2024-03-14T21:00:55.966 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-14T21:00:55.966 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405 --verbose --owner scheduled_yuriw@teuthology --name yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi --description fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-dbench} -- /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/orig.config.yaml 2024-03-14T21:00:56.012 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T21:00:56.097 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405 branch: wip-yuri4-testing-2024-03-13-0733-reef description: fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-dbench} email: yweinste@redhat.com first_in_suite: false job_id: '7600405' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri4-testing-2024-03-13-0733-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 234d532354068e06d01621fd032c3b663cead394 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 234d532354068e06d01621fd032c3b663cead394 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2024-03-13-0733-reef sha1: 234d532354068e06d01621fd032c3b663cead394 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 seed: 1956 sha1: 234d532354068e06d01621fd032c3b663cead394 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2024-03-13-0733-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_234d532354068e06d01621fd032c3b663cead394/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 234d532354068e06d01621fd032c3b663cead394 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIsFzqFfZuzGWdRz7NTpDc8OGmCraRPPVX0whB8j8nGlLW2CZFDOrfhSltO7o0I8PMZJ5UojN0FCwrTlpMIbjJA= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC8FSt3cq8kt44xgUYd26EGx1zagHHuP61WQJ0TefnB9toTrZAsEFMYq1EvrHvwIBAw+dx4UMwj6Tg5nT+YNDVY= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvawradYzO1Z9o2QO+uZ+V6JJs7yGcUFYNr2bd+3nXey6xgvubX6cevWGSStxIMBct5lZ/ACj4rJ4eA1VvS/8Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - workunit: clients: client.0: - fs/fscrypt.sh none dbench - fs/fscrypt.sh unlocked dbench timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-14_15:28:28 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-14T21:00:56.097 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_234d532354068e06d01621fd032c3b663cead394/qa; will attempt to use it 2024-03-14T21:00:56.098 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_234d532354068e06d01621fd032c3b663cead394/qa/tasks 2024-03-14T21:00:56.098 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-14T21:00:56.100 INFO:teuthology.task.internal:Checking packages... 2024-03-14T21:00:56.124 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '234d532354068e06d01621fd032c3b663cead394' 2024-03-14T21:00:56.124 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-14T21:00:56.124 INFO:teuthology.packaging:ref: None 2024-03-14T21:00:56.124 INFO:teuthology.packaging:tag: None 2024-03-14T21:00:56.124 INFO:teuthology.packaging:branch: wip-yuri4-testing-2024-03-13-0733-reef 2024-03-14T21:00:56.124 INFO:teuthology.packaging:sha1: 234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:00:56.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri4-testing-2024-03-13-0733-reef 2024-03-14T21:00:56.267 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-852.g234d5323 2024-03-14T21:00:56.268 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-14T21:00:56.274 INFO:teuthology.task.internal:no buildpackages task found 2024-03-14T21:00:56.274 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-14T21:00:56.280 INFO:teuthology.task.internal:Saving configuration 2024-03-14T21:00:56.295 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-14T21:00:56.304 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T21:00:56.329 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 20:54:34.964322', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIsFzqFfZuzGWdRz7NTpDc8OGmCraRPPVX0whB8j8nGlLW2CZFDOrfhSltO7o0I8PMZJ5UojN0FCwrTlpMIbjJA='} 2024-03-14T21:00:56.348 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 20:54:34.962660', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC8FSt3cq8kt44xgUYd26EGx1zagHHuP61WQJ0TefnB9toTrZAsEFMYq1EvrHvwIBAw+dx4UMwj6Tg5nT+YNDVY='} 2024-03-14T21:00:56.369 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 20:54:34.960779', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvawradYzO1Z9o2QO+uZ+V6JJs7yGcUFYNr2bd+3nXey6xgvubX6cevWGSStxIMBct5lZ/ACj4rJ4eA1VvS/8Y='} 2024-03-14T21:00:56.369 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-14T21:00:56.376 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-03-14T21:00:56.376 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-14T21:00:56.376 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['client.0'] 2024-03-14T21:00:56.376 INFO:teuthology.run_tasks:Running task console_log... 2024-03-14T21:00:56.477 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efd43f0b160>, signals=[15]) 2024-03-14T21:00:56.477 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-14T21:00:56.484 INFO:teuthology.task.internal:Opening connections... 2024-03-14T21:00:56.484 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T21:00:56.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:00:56.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T21:00:56.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:00:56.676 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-03-14T21:00:56.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:00:56.767 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-14T21:00:56.774 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-03-14T21:00:56.793 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-03-14T21:00:56.793 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-03-14T21:00:56.851 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T21:00:56.852 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-14T21:00:56.853 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-14T21:00:56.853 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-03-14T21:00:56.880 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-03-14T21:00:56.898 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-03-14T21:00:56.899 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-03-14T21:00:56.956 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T21:00:56.957 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-14T21:00:56.958 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-14T21:00:56.958 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-03-14T21:00:56.980 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-03-14T21:00:56.998 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-03-14T21:00:56.998 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-14T21:00:57.055 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-03-14T21:00:57.056 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T21:00:57.056 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-14T21:00:57.056 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-14T21:00:57.056 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-03-14T21:00:57.071 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-14T21:00:57.081 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-14T21:00:57.087 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-14T21:00:57.087 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-03-14T21:00:57.090 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-03-14T21:00:57.093 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-03-14T21:00:57.111 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-14T21:00:57.118 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-14T21:00:57.118 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-03-14T21:00:57.148 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-03-14T21:00:57.151 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-03-14T21:00:57.254 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-14T21:00:57.330 INFO:teuthology.run_tasks:Running task kernel... 2024-03-14T21:00:57.346 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-14T21:00:57.347 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-14T21:00:57.347 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-14T21:00:57.347 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-14T21:00:57.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T21:00:57.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-14T21:00:57.689 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-14T21:00:57.690 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3845b7ac715a3ff198582638d199a90e30e254da"... 2024-03-14T21:00:57.690 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-03-14T21:00:57.707 INFO:teuthology.orchestra.run.smithi163.stdout:5.14.0-361.el9.x86_64 2024-03-14T21:00:57.707 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-14T21:00:57.708 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-14T21:00:57.708 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum upgrade -y linux-firmware 2024-03-14T21:01:00.355 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - AppStream 7.1 MB/s | 6.7 MB 00:00 2024-03-14T21:01:01.955 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-14T21:01:03.831 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-03-14T21:01:07.246 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 19 MB 00:02 2024-03-14T21:01:13.438 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.1 MB 00:02 2024-03-14T21:01:14.955 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-03-14T21:01:20.240 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 4.0 MB/s | 21 MB 00:05 2024-03-14T21:01:25.172 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-03-14T21:01:27.482 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 1 Package 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:01:27.492 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 375 M 2024-03-14T21:01:27.493 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-03-14T21:01:34.840 INFO:teuthology.orchestra.run.smithi163.stdout:linux-firmware-20240219-143.el9.noarch.rpm 51 MB/s | 375 MB 00:07 2024-03-14T21:01:34.841 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-03-14T21:01:34.841 INFO:teuthology.orchestra.run.smithi163.stdout:Total 51 MB/s | 375 MB 00:07 2024-03-14T21:01:34.841 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-03-14T21:01:34.945 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-03-14T21:01:34.945 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-03-14T21:01:37.327 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-03-14T21:01:37.327 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-03-14T21:01:43.785 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-03-14T21:01:44.161 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-14T21:01:44.248 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-14T21:01:45.529 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-14T21:01:45.529 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-14T21:01:45.913 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-14T21:01:45.913 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:01:45.913 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-03-14T21:01:45.913 INFO:teuthology.orchestra.run.smithi163.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-14T21:01:45.913 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:01:45.913 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-03-14T21:01:46.224 INFO:teuthology.task.kernel:Downloading kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.0... 2024-03-14T21:01:46.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3845b7ac715a3ff198582638d199a90e30e254da 2024-03-14T21:01:46.367 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ 2024-03-14T21:01:46.368 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-14T21:01:46.371 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi163.front.sepia.ceph.com to complete... 2024-03-14T21:01:58.556 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-14 21:01:58 URL:https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm [336075563/336075563] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-14T21:01:58.556 INFO:teuthology.orchestra.run.smithi163.stderr:FINISHED --2024-03-14 21:01:58-- 2024-03-14T21:01:58.556 INFO:teuthology.orchestra.run.smithi163.stderr:Total wall clock time: 12s 2024-03-14T21:01:58.557 INFO:teuthology.orchestra.run.smithi163.stderr:Downloaded: 1 files, 321M in 12s (26.9 MB/s) 2024-03-14T21:01:58.559 INFO:teuthology.task.kernel:Installing kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.0... 2024-03-14T21:01:58.559 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-14T21:02:00.497 INFO:teuthology.orchestra.run.smithi163.stdout:Verifying... ######################################## 2024-03-14T21:02:00.572 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing... ######################################## 2024-03-14T21:02:00.652 INFO:teuthology.orchestra.run.smithi163.stdout:Updating / installing... 2024-03-14T21:02:08.185 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-6.8.0_rc7_g3845b7ac715a-1 ######################################## 2024-03-14T21:02:24.500 INFO:teuthology.orchestra.run.smithi163.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-14T21:02:47.063 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-14T21:02:47.064 DEBUG:teuthology.orchestra.run.smithi163:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-14T21:02:47.172 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a 2024-03-14T21:02:47.172 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/System.map 2024-03-14T21:02:47.172 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/config 2024-03-14T21:02:47.172 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel 2024-03-14T21:02:47.172 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch 2024-03-14T21:02:47.172 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86 2024-03-14T21:02:47.172 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-14T21:02:47.173 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-14T21:02:47.174 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ansi_cprng.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/anubis.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/arc4.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_memcpy.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_pq.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_tx.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_xor.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/raid6test.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authenc.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authencesn.ko 2024-03-14T21:02:47.175 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blake2b_generic.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_common.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_generic.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/camellia_generic.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast5_generic.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast6_generic.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast_common.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ccm.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cmac.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crc32_generic.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_engine.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_user.ko 2024-03-14T21:02:47.176 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cts.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/des_generic.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/dh_generic.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/echainiv.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/essiv.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/fcrypt.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/khazad.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4hc.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/md4.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/michael_mic.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcbc.ko 2024-03-14T21:02:47.177 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcrypt.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/rmd160.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/seed.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/serpent_generic.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tcrypt.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tea.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_common.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_generic.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/vmac.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/wp512.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xcbc.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xor.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xxhash_generic.ko 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers 2024-03-14T21:02:47.178 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_pad.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/custom_method.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/ec_sys.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/platform_profile.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbs.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbshc.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/video.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/acard-ahci.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ahci_platform.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ata_generic.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/libahci_platform.ko 2024-03-14T21:02:47.179 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_acpi.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ali.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_amd.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_artop.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atiixp.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atp867x.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd640.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd64x.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cypress.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_efar.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt366.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt37x.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it8213.ko 2024-03-14T21:02:47.180 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it821x.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_jmicron.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_marvell.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_mpiix.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_netcell.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ninja32.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87410.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87415.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_oldpiix.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_opti.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_optidma.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pcmcia.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-14T21:02:47.181 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_piccolo.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_rdc.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sch.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_serverworks.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sil680.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sis.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sl82c105.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_triflex.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_via.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pdc_adma.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_inic162x.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_mv.ko 2024-03-14T21:02:47.182 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_nv.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_promise.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_qstor.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil24.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sis.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_svw.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sx4.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_uli.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_via.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_vsc.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay/ks0108.ko 2024-03-14T21:02:47.183 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma/bcma.ko 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe/aoe.ko 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/brd.ko 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd/drbd.ko 2024-03-14T21:02:47.184 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/floppy.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/loop.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/nbd.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk/null_blk.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/pktcdvd.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/rbd.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/virtio_blk.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-14T21:02:47.185 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkfront.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram/zram.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hangcheck-timer.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/via-rng.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-14T21:02:47.186 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/lp.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave/mwave.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ppdev.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tlclk.ko 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm 2024-03-14T21:02:47.187 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/uv_mmtimer.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/virtio_console.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-14T21:02:47.188 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp.ko 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-aes.ko 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-sha.ko 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca 2024-03-14T21:02:47.189 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca/dca.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/amd64_edac.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/e752x_edac.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/edac_mce_amd.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3000_edac.ko 2024-03-14T21:02:47.190 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3200_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5100_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5400_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7300_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7core_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i82975x_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/ie31200_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/sb_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/x38_edac.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/edd.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-ich.ko 2024-03-14T21:02:47.191 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-appleir.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-aureal.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-axff.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-dr.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elecom.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elo.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-emsff.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gaff.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gt683r.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gyration.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-14T21:02:47.192 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtekff.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-hyperv.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-icade.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-keytouch.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-kye.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lcpower.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-led.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lenovo.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-multitouch.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-ortek.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-penmount.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-petalynx.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-picolcd.ko 2024-03-14T21:02:47.193 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-pl.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-plantronics.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-primax.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-rmi.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-common.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-14T21:02:47.194 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-saitek.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-samsung.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sjoy.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sony.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-speedlink.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-steelseries.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sunplus.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tivo.ko 2024-03-14T21:02:47.195 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tmff.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-topseed.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-twinhan.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-uclogic.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-waltop.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-wiimote.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-xinmo.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zpff.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zydacron.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/uhid.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/wacom.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_balloon.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_utils.ko 2024-03-14T21:02:47.196 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_vmbus.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru3.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7414.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7418.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adc128d818.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1025.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1026.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1029.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1031.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm9240.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ads7828.ko 2024-03-14T21:02:47.197 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7410.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7411.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7462.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7470.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7475.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7x10.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/amc6821.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/applesmc.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asb100.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asc7621.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/atxp1.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/coretemp.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-14T21:02:47.198 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dme1737.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds1621.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds620.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc1403.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc6w201.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71805f.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71882fg.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f75375s.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fam15h_power.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fschmd.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g760a.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g762.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl518sm.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl520sm.ko 2024-03-14T21:02:47.199 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5500_temp.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5k_amb.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmaem.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmpex.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina209.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina2xx.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/it87.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k10temp.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k8temp.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lineage-pem.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm63.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm73.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm75.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm77.ko 2024-03-14T21:02:47.200 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm78.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm80.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm83.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm85.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm87.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm90.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm92.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm93.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95234.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95241.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95245.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc2945.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4151.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4215.ko 2024-03-14T21:02:47.201 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4222.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4245.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4260.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4261.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max16065.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1619.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1668.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max197.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6639.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6650.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6697.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/mcp3021.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6683.ko 2024-03-14T21:02:47.202 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775-core.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct7802.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87360.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87427.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pcf8591.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-14T21:02:47.203 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/powr1220.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5627.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5636.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht15.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht21.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/shtc1.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sis5595.ko 2024-03-14T21:02:47.204 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47b397.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m1.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m192.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/thmc50.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp102.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp103.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp401.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp421.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via-cputemp.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via686a.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt1211.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt8231.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627ehf.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627hf.ko 2024-03-14T21:02:47.205 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83781d.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83791d.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83792d.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83793.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83795.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l785ts.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l786ng.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-14T21:02:47.206 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-14T21:02:47.207 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-14T21:02:47.208 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-dev.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-mux.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-smbus.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-stub.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_core.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-14T21:02:47.209 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib 2024-03-14T21:02:47.210 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/gameport.ko 2024-03-14T21:02:47.211 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/joydev.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/appletouch.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/bcm5974.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/cyapatp.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/sermouse.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio 2024-03-14T21:02:47.212 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/altera_ps2.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/arc_ps2.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/serio_raw.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/sparse-keymap.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-blinkm.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lm3530.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lp3944.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-ss4200.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-14T21:02:47.213 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache/bcache.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-bio-prison.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache-smq.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-crypt.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-delay.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-flakey.ko 2024-03-14T21:02:47.214 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-log-userspace.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-multipath.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-queue-length.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-raid.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-round-robin.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-service-time.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-switch.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-thin-pool.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-verity.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid0.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid1.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid10.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid456.ko 2024-03-14T21:02:47.215 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptbase.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptctl.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptfc.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptlan.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptsas.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptscsih.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptspi.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_ich.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_sch.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/mfd-core.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/sm501.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/viperboard.ko 2024-03-14T21:02:47.216 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/vx855.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/wl1273-core.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds9802als.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds990x.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/bh1770glc.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710/cb710.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/at24.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/max6875.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/enclosure.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/hpilo.ko 2024-03-14T21:02:47.217 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29003.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29020.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-me.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-txe.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-14T21:02:47.218 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_7xx1.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_core.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tsl2550.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_balloon.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding/bonding.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-14T21:02:47.219 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dummy.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/eql.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec 2024-03-14T21:02:47.220 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-14T21:02:47.221 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade 2024-03-14T21:02:47.222 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic 2024-03-14T21:02:47.223 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dnet.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-14T21:02:47.224 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ethoc.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/fealnx.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-14T21:02:47.225 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/jme.ko 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-14T21:02:47.226 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-14T21:02:47.227 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-14T21:02:47.228 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-14T21:02:47.229 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti 2024-03-14T21:02:47.230 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/geneve.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ifb.ko 2024-03-14T21:02:47.231 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvlan.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvtap.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mii.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/net_failover.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/netconsole.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/nlmon.ko 2024-03-14T21:02:47.232 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/amd.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm87xx.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/broadcom.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/cicada.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/davicom.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/dp83640.ko 2024-03-14T21:02:47.233 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/et1011c.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/icplus.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/lxt.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/marvell.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/micrel.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/national.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/phylink.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/qsemi.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/realtek.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/smsc.ko 2024-03-14T21:02:47.234 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/ste10Xp.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/vitesse.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_async.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppoe.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppox.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pptp.ko 2024-03-14T21:02:47.235 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slhc.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slip.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/sungem_phy.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tap.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_random.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-14T21:02:47.236 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tun.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/veth.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/virtio_net.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan/vxlan.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netfront.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb/ntb.ko 2024-03-14T21:02:47.237 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme-core.ko 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme.ko 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport.ko 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_cs.ko 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_pc.ko 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_serial.ko 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-14T21:02:47.238 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/xen-pcifront.ko 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/i82092.ko 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/pd6729.ko 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-14T21:02:47.239 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acerhdf.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/hdaps.ko 2024-03-14T21:02:47.240 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel_ips.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi.ko 2024-03-14T21:02:47.241 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps_parport.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/pps_core.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp_kvm.ko 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc 2024-03-14T21:02:47.242 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-em3027.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-14T21:02:47.243 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-max6900.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-14T21:02:47.244 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-x1205.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-9xxx.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-sas.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-xxxx.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/BusLogic.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/a100u2w.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/advansys.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/am53c974.ko 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr 2024-03-14T21:02:47.245 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/atp870u.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa/bfa.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ch.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi 2024-03-14T21:02:47.246 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dc395x.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dmx3191d.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esp_scsi.ko 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe 2024-03-14T21:02:47.247 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic/fnic.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hpsa.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hptiop.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hv_storvsc.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/initio.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ipr.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ips.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci/isci.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-14T21:02:47.248 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc/libfc.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas/libsas.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-14T21:02:47.249 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvumi.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pmcraid.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla1280.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/raid_class.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_debug.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-14T21:02:47.250 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ses.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/st.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/stex.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/virtio_scsi.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/wd719x.ko 2024-03-14T21:02:47.251 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb/ssb.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_file.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_iblock.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_mod.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_pscsi.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_user.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-14T21:02:47.252 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-14T21:02:47.253 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_gsm.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_hdlc.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/nozomi.ko 2024-03-14T21:02:47.254 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/arc_uart.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_aec.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_cif.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_pci_generic.ko 2024-03-14T21:02:47.255 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_sercos3.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-acm.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usblp.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usbtmc.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/mdc800.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/microtek.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/adutux.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/appledisplay.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi26.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi62.ko 2024-03-14T21:02:47.256 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ezusb.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/idmouse.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/iowarrior.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ldusb.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/legousbtower.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usb3503.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usblcd.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/uss720.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/yurex.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy/phy-generic.ko 2024-03-14T21:02:47.257 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/aircable.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ark3116.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ch341.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cp210x.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cyberjack.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/empeg.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_ti.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipaq.ko 2024-03-14T21:02:47.258 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipw.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ir-usb.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mct_u232.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7720.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7840.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/navman.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/omninet.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/opticon.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/option.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/oti6858.ko 2024-03-14T21:02:47.259 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/pl2303.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcaux.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcserial.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/quatech2.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/safe_serial.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/sierra.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ssu100.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/symbolserial.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_debug.ko 2024-03-14T21:02:47.260 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/visor.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/whiteheat.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/uas.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-karma.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-14T21:02:47.261 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/usb-storage.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-14T21:02:47.262 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_net.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_scsi.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/apple_bl.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/lcd.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-14T21:02:47.263 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_balloon.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_mmio.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_ring.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2482.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2490.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-14T21:02:47.264 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/wire.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-14T21:02:47.265 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/hpwdt.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/i6300esb.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ib700wdt.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ibmasr.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it87_wdt.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/machzwd.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/nv_tco.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-14T21:02:47.266 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/softdog.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/via_wdt.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/wdt_pci.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/xen_wdt.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-evtchn.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntalloc.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntdev.ko 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback 2024-03-14T21:02:47.267 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-privcmd.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p/9p.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs/affs.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs/befs.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/binfmt_misc.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs/btrfs.ko 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles 2024-03-14T21:02:47.268 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles/cachefiles.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph/ceph.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda/coda.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs/cramfs.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm/dlm.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs/f2fs.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/fat.ko 2024-03-14T21:02:47.269 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/msdos.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/vfat.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/cuse.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/fuse.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2/gfs2.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs/hfs.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus/hfsplus.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs/isofs.ko 2024-03-14T21:02:47.270 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs/jfs.ko 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd/lockd.ko 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix/minix.ko 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs/netfs.ko 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-14T21:02:47.271 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfs.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv3.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv4.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/grace.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/nfs_acl.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd/nfsd.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2/nilfs2.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-celtic.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-centeuro.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-croatian.ko 2024-03-14T21:02:47.272 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-cyrillic.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-gaelic.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-greek.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-iceland.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-inuit.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-roman.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-romanian.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-turkish.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1250.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1251.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1255.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp737.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp775.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp850.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp852.ko 2024-03-14T21:02:47.273 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp855.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp857.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp860.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp861.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp862.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp863.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp864.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp865.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp866.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp869.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp874.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp932.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp936.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp949.ko 2024-03-14T21:02:47.274 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp950.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_euc-jp.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-1.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-13.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-14.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-15.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-2.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-3.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-4.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-5.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-6.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-7.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-9.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-r.ko 2024-03-14T21:02:47.275 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-ru.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-u.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_utf8.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-14T21:02:47.276 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs/overlay.ko 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore/ramoops.ko 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs/reiserfs.ko 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs/romfs.ko 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client/cifs.ko 2024-03-14T21:02:47.277 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_arc4.ko 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_md4.ko 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs/squashfs.ko 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv/sysv.ko 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf/udf.ko 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs/ufs.ko 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs/xfs.ko 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel 2024-03-14T21:02:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu/rcutorture.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/torture.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/asn1_encoder.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc-itu-t.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc8.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libarc4.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libdes.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/libcrc32c.ko 2024-03-14T21:02:47.279 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lru_cache.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4_compress.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4hc_compress.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math/cordic.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6/raid6_pq.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_bm.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_fsm.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_kmp.ko 2024-03-14T21:02:47.280 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd/zstd_compress.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm/hwpoison-inject.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/6lowpan.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_dest.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_fragment.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_hop.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_mobility.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_routing.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_udp.ko 2024-03-14T21:02:47.281 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/garp.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/mrp.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/p8022.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/psnap.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/stp.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q/8021q.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_fd.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_rdma.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_virtio.ko 2024-03-14T21:02:47.282 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv/batman-adv.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/br_netfilter.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/bridge.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-14T21:02:47.283 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtables.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-14T21:02:47.284 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph/libceph.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/failover.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/pktgen.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_diag.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv4.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv6.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver/dns_resolver.ko 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa 2024-03-14T21:02:47.285 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/dsa_core.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_brcm.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_dsa.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_none.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_trailer.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ah4.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/esp4.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/fou.ko 2024-03-14T21:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/gre.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/inet_diag.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_gre.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_tunnel.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_vti.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipcomp.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipip.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-14T21:02:47.287 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_bic.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_dctcp.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_diag.ko 2024-03-14T21:02:47.288 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_highspeed.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_htcp.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_hybla.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_illinois.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_lp.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_scalable.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_vegas.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_veno.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_westwood.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_yeah.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tunnel4.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_diag.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_tunnel.ko 2024-03-14T21:02:47.289 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ah6.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/esp6.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/fou6.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_tunnel.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_vti.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ipcomp6.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-14T21:02:47.290 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-14T21:02:47.291 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/sit.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/tunnel6.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key/af_key.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_core.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_eth.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip6.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_netlink.ko 2024-03-14T21:02:47.292 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ppp.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc/llc.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154/mac802154.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls/mpls_gso.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-14T21:02:47.293 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-14T21:02:47.294 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conncount.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-14T21:02:47.295 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_log_syslog.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-14T21:02:47.296 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_irc.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_sip.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_tables.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_log.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_chain_nat.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_compat.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_ct.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fib.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-14T21:02:47.297 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_hash.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_limit.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_log.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_masq.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_nat.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_numgen.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_queue.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_quota.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_redir.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject_inet.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_socket.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_tunnel.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_AUDIT.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-14T21:02:47.298 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CT.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_DSCP.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HL.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HMARK.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LED.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LOG.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NETMAP.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFLOG.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_RATEEST.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_SECMARK.ko 2024-03-14T21:02:47.299 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TEE.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TPROXY.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TRACE.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_addrtype.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_bpf.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cgroup.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cluster.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_comment.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connbytes.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlabel.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlimit.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connmark.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_conntrack.ko 2024-03-14T21:02:47.300 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cpu.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dccp.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_devgroup.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dscp.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ecn.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_esp.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hashlimit.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_helper.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hl.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipcomp.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_iprange.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipvs.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_l2tp.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_length.ko 2024-03-14T21:02:47.301 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_limit.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mac.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mark.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_multiport.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nat.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nfacct.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_osf.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_owner.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_physdev.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_pkttype.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_policy.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_quota.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_rateest.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_realm.ko 2024-03-14T21:02:47.302 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_recent.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_sctp.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_set.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_socket.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_state.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_statistic.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_string.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_tcpmss.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_time.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_u32.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink/netlink_diag.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh/nsh.ko 2024-03-14T21:02:47.303 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/openvswitch.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-geneve.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-gre.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-vxlan.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet/af_packet_diag.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_rdma.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_tcp.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_csum.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_gact.ko 2024-03-14T21:02:47.304 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_mirred.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_nat.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_pedit.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_police.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_simple.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_skbedit.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_vlan.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_basic.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_bpf.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_flow.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_fw.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_route.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_u32.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_cmp.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_ipset.ko 2024-03-14T21:02:47.305 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_meta.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_nbyte.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_text.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_u32.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_choke.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_codel.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_drr.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_fq.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_gred.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hfsc.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hhf.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_htb.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_ingress.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio.ko 2024-03-14T21:02:47.306 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio_lib.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_multiq.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_netem.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_pie.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_plug.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_prio.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_qfq.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_red.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfb.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfq.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_tbf.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_teql.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp_diag.ko 2024-03-14T21:02:47.307 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/sunrpc.ko 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/diag.ko 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/tipc.ko 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix/unix_diag.ko 2024-03-14T21:02:47.308 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys/trusted.ko 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt 2024-03-14T21:02:47.309 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib/irqbypass.ko 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias.bin 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.alias.bin 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.bin 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.modinfo 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep.bin 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.devname 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.order 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.softdep 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols 2024-03-14T21:02:47.310 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols.bin 2024-03-14T21:02:47.311 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-14T21:02:47.315 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-g3845b7ac715a 2024-03-14T21:02:47.315 DEBUG:teuthology.orchestra.run.smithi163:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-14T21:02:47.337 INFO:teuthology.orchestra.run.smithi163.stdout:postinstall scriptlet (using /bin/sh): 2024-03-14T21:02:47.337 INFO:teuthology.orchestra.run.smithi163.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-14T21:02:47.337 INFO:teuthology.orchestra.run.smithi163.stdout: /usr/bin/kernel-install add 6.8.0-rc7-g3845b7ac715a /lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-14T21:02:47.337 INFO:teuthology.orchestra.run.smithi163.stdout:fi 2024-03-14T21:02:47.337 INFO:teuthology.orchestra.run.smithi163.stdout:for file in vmlinuz System.map config; do 2024-03-14T21:02:47.337 INFO:teuthology.orchestra.run.smithi163.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a"; then 2024-03-14T21:02:47.337 INFO:teuthology.orchestra.run.smithi163.stdout: cp "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a" 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout: fi 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:done 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:new-kernel-pkg --remove 6.8.0-rc7-g3845b7ac715a --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-g3845b7ac715a.img 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-install remove 6.8.0-rc7-g3845b7ac715a 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:fi 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-g3845b7ac715a 2024-03-14T21:02:47.338 INFO:teuthology.orchestra.run.smithi163.stdout:fi 2024-03-14T21:02:47.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -qi grub2-tools 2024-03-14T21:02:47.423 INFO:teuthology.orchestra.run.smithi163.stdout:Name : grub2-tools 2024-03-14T21:02:47.423 INFO:teuthology.orchestra.run.smithi163.stdout:Epoch : 1 2024-03-14T21:02:47.423 INFO:teuthology.orchestra.run.smithi163.stdout:Version : 2.06 2024-03-14T21:02:47.423 INFO:teuthology.orchestra.run.smithi163.stdout:Release : 68.el9 2024-03-14T21:02:47.423 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: x86_64 2024-03-14T21:02:47.423 INFO:teuthology.orchestra.run.smithi163.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Group : Unspecified 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Size : 8264254 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:License : GPLv3+ 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Packager : builder@centos.org 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Vendor : CentOS 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Summary : Support tools for GRUB. 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:Description : 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-14T21:02:47.424 INFO:teuthology.orchestra.run.smithi163.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-14T21:02:47.425 INFO:teuthology.orchestra.run.smithi163.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-14T21:02:47.425 INFO:teuthology.orchestra.run.smithi163.stdout:hardware devices. 2024-03-14T21:02:47.425 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:02:47.425 INFO:teuthology.orchestra.run.smithi163.stdout:This subpackage provides tools for support of all platforms. 2024-03-14T21:02:47.426 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-14T21:02:47.426 INFO:teuthology.task.kernel:Updating grub on smithi163 to boot 6.8.0-rc7-g3845b7ac715a 2024-03-14T21:02:47.426 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-14T21:02:48.089 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2024-03-14T21:02:49.573 INFO:teuthology.orchestra.run.smithi163.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-14T21:02:49.604 INFO:teuthology.orchestra.run.smithi163.stderr:done 2024-03-14T21:02:49.606 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-03-14T21:02:49.621 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.URuELhKfA2 2024-03-14T21:02:49.621 DEBUG:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.URuELhKfA2 2024-03-14T21:02:49.690 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.URuELhKfA2 2024-03-14T21:02:49.838 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.URuELhKfA2 is 6KB 2024-03-14T21:02:49.886 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.URuELhKfA2 2024-03-14T21:02:49.900 DEBUG:teuthology.orchestra.run.smithi163:> sudo /bin/ls /boot/loader/entries || true 2024-03-14T21:02:49.965 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-14T21:02:49.965 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-14T21:02:49.965 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-14T21:02:49.965 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a.conf 2024-03-14T21:02:49.967 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a 2024-03-14T21:02:50.105 DEBUG:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2024-03-14T21:03:20.108 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-14T21:03:20.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:03:20.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:03:38.530 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-14T21:03:47.532 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:03:47.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:03:50.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-14T21:04:02.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:04:02.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:04:05.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-14T21:04:20.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:04:20.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:04:39.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-14T21:04:57.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:04:57.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:04:57.682 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-03-14T21:04:58.303 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:04:58.303 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-g3845b7ac715a"... 2024-03-14T21:04:58.303 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-03-14T21:04:58.344 INFO:teuthology.orchestra.run.smithi163.stdout:6.8.0-rc7-g3845b7ac715a 2024-03-14T21:04:58.344 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-g3845b7ac715a vs 6.8.0-rc7-g3845b7ac715a 2024-03-14T21:04:58.344 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-14T21:04:58.344 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-14T21:04:59.346 DEBUG:teuthology.parallel:result is None 2024-03-14T21:04:59.346 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-14T21:04:59.353 INFO:teuthology.task.internal:Creating test directory... 2024-03-14T21:04:59.354 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-14T21:04:59.356 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-14T21:04:59.359 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-14T21:04:59.412 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-14T21:04:59.419 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-14T21:04:59.425 INFO:teuthology.task.internal:Creating archive directory... 2024-03-14T21:04:59.426 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-14T21:04:59.428 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-14T21:04:59.430 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-14T21:04:59.686 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-14T21:04:59.693 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-14T21:04:59.693 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-14T21:04:59.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T21:04:59.710 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-14T21:04:59.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T21:04:59.727 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-14T21:04:59.753 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-14T21:04:59.769 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-14T21:04:59.811 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T21:04:59.829 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T21:04:59.831 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T21:04:59.859 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T21:04:59.903 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T21:04:59.979 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T21:04:59.983 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-14T21:04:59.991 INFO:teuthology.task.internal:Configuring sudo... 2024-03-14T21:04:59.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-14T21:04:59.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-14T21:04:59.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-14T21:05:00.067 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-14T21:05:00.076 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-14T21:05:00.076 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-14T21:05:00.079 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-14T21:05:00.081 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-14T21:05:00.122 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T21:05:00.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T21:05:00.234 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T21:05:00.319 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T21:05:00.365 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:05:00.365 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-14T21:05:00.437 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T21:05:00.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T21:05:00.534 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T21:05:00.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T21:05:00.665 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:05:00.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-14T21:05:00.735 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T21:05:00.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T21:05:00.895 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T21:05:00.968 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T21:05:01.044 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:05:01.044 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-14T21:05:01.162 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-03-14T21:05:01.167 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-03-14T21:05:01.170 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-03-14T21:05:01.207 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T21:05:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T21:05:01.302 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T21:05:01.703 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-14T21:05:01.711 INFO:teuthology.task.internal:Starting timer... 2024-03-14T21:05:01.711 INFO:teuthology.run_tasks:Running task pcp... 2024-03-14T21:05:01.721 INFO:teuthology.run_tasks:Running task selinux... 2024-03-14T21:05:01.732 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-03-14T21:05:01.801 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-03-14T21:05:01.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-03-14T21:05:01.874 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-03-14T21:05:01.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-03-14T21:05:02.006 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-03-14T21:05:02.011 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-14T21:05:02.012 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-03-14T21:05:02.035 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-03-14T21:05:02.036 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-03-14T21:05:02.057 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-03-14T21:05:02.057 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-03-14T21:05:02.129 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-03-14T21:05:02.129 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2024-03-14T21:05:02.130 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T21:05:02.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T21:05:02.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T21:05:02.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T21:05:02.197 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T21:05:02.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T21:05:02.275 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-14T21:05:02.275 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-03-14T21:05:02.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-03-14T21:05:02.359 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-03-14T21:05:02.458 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-14T21:05:02.468 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-14T21:05:02.485 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-14T21:05:02.486 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-14T21:10:55.171 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-03-14T21:10:55.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-14T21:10:55.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:10:55.285 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-03-14T21:10:55.357 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-14T21:10:55.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-14T21:10:55.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:10:55.444 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-14T21:10:55.469 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-14T21:10:55.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:10:55.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T21:10:55.621 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-03-14T21:10:55.770 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-14T21:10:55.770 INFO:teuthology.run_tasks:Running task clock... 2024-03-14T21:10:55.785 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-14T21:10:55.785 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-14T21:10:55.786 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T21:10:55.788 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-14T21:10:55.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T21:10:55.791 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-14T21:10:55.791 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T21:10:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-14T21:10:55.821 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-14T21:10:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-14T21:10:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-14T21:10:55.868 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-03-14T21:10:55.878 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-03-14T21:10:55.881 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-03-14T21:10:55.893 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-14T21:10:55.896 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-03-14T21:10:55.898 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-14T21:10:55.914 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-14T21:10:55.915 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-14T21:10:55.934 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-14T21:10:55.946 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-14T21:10:55.958 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-14T21:10:55.961 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T21:10:55.961 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-14T21:10:55.961 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:55.961 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:55.961 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:55.961 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-14T21:10:55.986 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T21:10:55.986 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-14T21:10:55.986 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:55.986 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:55.986 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:55.986 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-03-14T21:10:56.079 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-03-14T21:10:56.132 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-14T21:10:56.184 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-14T21:10:56.309 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-03-14T21:10:56.318 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T21:10:56.318 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-03-14T21:10:56.319 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:56.319 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:56.319 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:56.319 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T21:10:56.320 INFO:teuthology.run_tasks:Running task install... 2024-03-14T21:10:56.330 DEBUG:teuthology.task.install:project ceph 2024-03-14T21:10:56.331 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '234d532354068e06d01621fd032c3b663cead394'}} 2024-03-14T21:10:56.331 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '234d532354068e06d01621fd032c3b663cead394'} 2024-03-14T21:10:56.331 INFO:teuthology.task.install:Using flavor: default 2024-03-14T21:10:56.337 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-14T21:10:56.337 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-14T21:10:56.338 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '234d532354068e06d01621fd032c3b663cead394', 'tag': None, 'wait_for_package': False} 2024-03-14T21:10:56.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:10:56.340 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '234d532354068e06d01621fd032c3b663cead394', 'tag': None, 'wait_for_package': False} 2024-03-14T21:10:56.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:10:56.341 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '234d532354068e06d01621fd032c3b663cead394', 'tag': None, 'wait_for_package': False} 2024-03-14T21:10:56.341 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:10:56.487 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/ 2024-03-14T21:10:56.487 INFO:teuthology.task.install.rpm:Package version is 18.2.2-852.g234d5323 2024-03-14T21:10:56.556 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/ 2024-03-14T21:10:56.556 INFO:teuthology.task.install.rpm:Package version is 18.2.2-852.g234d5323 2024-03-14T21:10:56.562 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/ 2024-03-14T21:10:56.563 INFO:teuthology.task.install.rpm:Package version is 18.2.2-852.g234d5323 2024-03-14T21:10:56.660 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-14T21:10:56.661 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:10:56.661 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-14T21:10:56.695 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-14T21:10:56.696 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/234d532354068e06d01621fd032c3b663cead394/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-14T21:10:56.728 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-14T21:10:56.728 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:10:56.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-14T21:10:56.733 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-03-13-0733-reef/234d532354068e06d01621fd032c3b663cead394/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-14T21:10:56.733 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:10:56.733 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-14T21:10:56.757 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-14T21:10:56.757 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/234d532354068e06d01621fd032c3b663cead394/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-14T21:10:56.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-14T21:10:56.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-14T21:10:56.835 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-14T21:10:56.836 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/234d532354068e06d01621fd032c3b663cead394/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-14T21:10:56.858 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-14T21:10:56.888 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-03-14T21:10:56.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-03-14T21:10:56.909 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-14T21:10:56.974 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-14T21:10:56.975 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-03-14T21:10:56.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-03-14T21:10:57.107 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-14T21:10:57.138 INFO:teuthology.orchestra.run.smithi049.stdout:74 files removed 2024-03-14T21:10:57.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-14T21:10:57.197 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-03-14T21:10:57.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-03-14T21:10:57.212 INFO:teuthology.orchestra.run.smithi097.stdout:74 files removed 2024-03-14T21:10:57.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-14T21:10:57.571 INFO:teuthology.orchestra.run.smithi163.stdout:74 files removed 2024-03-14T21:10:57.613 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-14T21:10:57.908 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 177 kB/s | 77 kB 00:00 2024-03-14T21:10:58.029 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 154 kB/s | 77 kB 00:00 2024-03-14T21:10:58.275 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 30 kB/s | 9.9 kB 00:00 2024-03-14T21:10:58.446 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 26 kB/s | 9.9 kB 00:00 2024-03-14T21:10:58.576 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 169 kB/s | 77 kB 00:00 2024-03-14T21:10:58.634 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-03-14T21:10:58.763 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-03-14T21:10:59.037 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-14T21:10:59.110 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 22 kB/s | 9.9 kB 00:00 2024-03-14T21:10:59.215 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-14T21:10:59.610 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-03-14T21:11:00.771 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-14T21:11:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-14T21:11:00.996 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - AppStream 5.1 MB/s | 6.7 MB 00:01 2024-03-14T21:11:01.906 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-14T21:11:02.614 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-14T21:11:03.751 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-14T21:11:04.924 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 19 MB 00:01 2024-03-14T21:11:08.693 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 19 MB 00:03 2024-03-14T21:11:09.666 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.1 MB 00:01 2024-03-14T21:11:10.827 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 846 kB/s | 8.0 MB 00:09 2024-03-14T21:11:11.126 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 15 kB 00:00 2024-03-14T21:11:12.588 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-14T21:11:15.277 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.1 MB 00:02 2024-03-14T21:11:17.011 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 15 kB 00:00 2024-03-14T21:11:17.486 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-14T21:11:18.676 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-14T21:11:19.974 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T21:11:19.975 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T21:11:19.987 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-14T21:11:19.988 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-14T21:11:19.989 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-14T21:11:19.990 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-14T21:11:19.991 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-14T21:11:19.994 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-14T21:11:19.995 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-14T21:11:20.071 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-14T21:11:20.083 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================== 2024-03-14T21:11:20.083 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-14T21:11:20.083 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================== 2024-03-14T21:11:20.083 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-14T21:11:20.083 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-14T21:11:20.083 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:18.2.2-852.g234d5323.el9 ceph 6.5 k 2024-03-14T21:11:20.083 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:18.2.2-852.g234d5323.el9 ceph 5.2 M 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:18.2.2-852.g234d5323.el9 ceph 843 k 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-852.g234d5323.el9 ceph 142 k 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:18.2.2-852.g234d5323.el9 ceph 1.4 M 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 138 k 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 1.7 M 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 7.4 M 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 49 k 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:18.2.2-852.g234d5323.el9 ceph 7.7 M 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:18.2.2-852.g234d5323.el9 ceph 40 M 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 264 k 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 224 k 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:18.2.2-852.g234d5323.el9 ceph 222 k 2024-03-14T21:11:20.084 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 18 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:18.2.2-852.g234d5323.el9 ceph 31 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 659 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:18.2.2-852.g234d5323.el9 ceph 127 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-14T21:11:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:18.2.2-852.g234d5323.el9 ceph 161 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:18.2.2-852.g234d5323.el9 ceph 321 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:18.2.2-852.g234d5323.el9 ceph 298 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:18.2.2-852.g234d5323.el9 ceph 99 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:18.2.2-852.g234d5323.el9 ceph 87 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.0 M 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:18.2.2-852.g234d5323.el9 ceph 172 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-14T21:11:20.086 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.3 M 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.0 M 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-14T21:11:20.087 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:18.2.2-852.g234d5323.el9 ceph 18 M 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 23 k 2024-03-14T21:11:20.088 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:18.2.2-852.g234d5323.el9 ceph 2.1 M 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 245 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:18.2.2-852.g234d5323.el9 ceph 4.5 M 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:18.2.2-852.g234d5323.el9 ceph 18 M 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 15 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:18.2.2-852.g234d5323.el9 ceph 25 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-14T21:11:20.089 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:18.2.2-852.g234d5323.el9 ceph 166 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:18.2.2-852.g234d5323.el9 ceph 474 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 4.5 M 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-14T21:11:20.090 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:18.2.2-852.g234d5323.el9 ceph 45 k 2024-03-14T21:11:20.091 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:18.2.2-852.g234d5323.el9 ceph 127 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-14T21:11:20.092 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-14T21:11:20.093 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-14T21:11:20.094 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-14T21:11:20.095 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================== 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout:Install 132 Packages 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 24 Packages 2024-03-14T21:11:20.096 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:11:20.097 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 205 M 2024-03-14T21:11:20.097 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-14T21:11:20.495 INFO:teuthology.orchestra.run.smithi097.stdout:(1/156): ceph-18.2.2-852.g234d5323.el9.x86_64.r 38 kB/s | 6.5 kB 00:00 2024-03-14T21:11:20.721 INFO:teuthology.orchestra.run.smithi097.stdout:(2/156): ceph-fuse-18.2.2-852.g234d5323.el9.x86 3.7 MB/s | 843 kB 00:00 2024-03-14T21:11:20.779 INFO:teuthology.orchestra.run.smithi097.stdout:(3/156): ceph-immutable-object-cache-18.2.2-852 2.4 MB/s | 142 kB 00:00 2024-03-14T21:11:21.146 INFO:teuthology.orchestra.run.smithi097.stdout:(4/156): ceph-mds-18.2.2-852.g234d5323.el9.x86_ 5.8 MB/s | 2.1 MB 00:00 2024-03-14T21:11:21.397 INFO:teuthology.orchestra.run.smithi097.stdout:(5/156): ceph-mgr-18.2.2-852.g234d5323.el9.x86_ 5.8 MB/s | 1.4 MB 00:00 2024-03-14T21:11:21.680 INFO:teuthology.orchestra.run.smithi097.stdout:(6/156): ceph-common-18.2.2-852.g234d5323.el9.x 14 MB/s | 18 MB 00:01 2024-03-14T21:11:22.047 INFO:teuthology.orchestra.run.smithi097.stdout:(7/156): ceph-mon-18.2.2-852.g234d5323.el9.x86_ 6.9 MB/s | 4.5 MB 00:00 2024-03-14T21:11:22.214 INFO:teuthology.orchestra.run.smithi097.stdout:(8/156): ceph-base-18.2.2-852.g234d5323.el9.x86 2.7 MB/s | 5.2 MB 00:01 2024-03-14T21:11:22.314 INFO:teuthology.orchestra.run.smithi097.stdout:(9/156): ceph-selinux-18.2.2-852.g234d5323.el9. 252 kB/s | 25 kB 00:00 2024-03-14T21:11:22.798 INFO:teuthology.orchestra.run.smithi097.stdout:(10/156): ceph-osd-18.2.2-852.g234d5323.el9.x86 16 MB/s | 18 MB 00:01 2024-03-14T21:11:22.856 INFO:teuthology.orchestra.run.smithi097.stdout:(11/156): cephfs-mirror-18.2.2-852.g234d5323.el 3.7 MB/s | 222 kB 00:00 2024-03-14T21:11:22.923 INFO:teuthology.orchestra.run.smithi097.stdout:(12/156): libcephfs-devel-18.2.2-852.g234d5323. 466 kB/s | 31 kB 00:00 2024-03-14T21:11:23.032 INFO:teuthology.orchestra.run.smithi097.stdout:(13/156): ceph-radosgw-18.2.2-852.g234d5323.el9 7.8 MB/s | 7.7 MB 00:00 2024-03-14T21:11:23.065 INFO:teuthology.orchestra.run.smithi097.stdout:(14/156): libcephfs2-18.2.2-852.g234d5323.el9.x 4.5 MB/s | 659 kB 00:00 2024-03-14T21:11:23.091 INFO:teuthology.orchestra.run.smithi097.stdout:(15/156): libcephsqlite-18.2.2-852.g234d5323.el 2.8 MB/s | 166 kB 00:00 2024-03-14T21:11:23.116 INFO:teuthology.orchestra.run.smithi097.stdout:(16/156): librados-devel-18.2.2-852.g234d5323.e 2.5 MB/s | 127 kB 00:00 2024-03-14T21:11:23.167 INFO:teuthology.orchestra.run.smithi097.stdout:(17/156): libradosstriper1-18.2.2-852.g234d5323 6.1 MB/s | 474 kB 00:00 2024-03-14T21:11:23.267 INFO:teuthology.orchestra.run.smithi097.stdout:(18/156): python3-ceph-argparse-18.2.2-852.g234 450 kB/s | 45 kB 00:00 2024-03-14T21:11:23.359 INFO:teuthology.orchestra.run.smithi097.stdout:(19/156): python3-ceph-common-18.2.2-852.g234d5 1.4 MB/s | 127 kB 00:00 2024-03-14T21:11:23.434 INFO:teuthology.orchestra.run.smithi097.stdout:(20/156): librgw2-18.2.2-852.g234d5323.el9.x86_ 14 MB/s | 4.5 MB 00:00 2024-03-14T21:11:23.459 INFO:teuthology.orchestra.run.smithi097.stdout:(21/156): python3-cephfs-18.2.2-852.g234d5323.e 1.6 MB/s | 161 kB 00:00 2024-03-14T21:11:23.493 INFO:teuthology.orchestra.run.smithi097.stdout:(22/156): python3-rados-18.2.2-852.g234d5323.el 5.4 MB/s | 321 kB 00:00 2024-03-14T21:11:23.535 INFO:teuthology.orchestra.run.smithi097.stdout:(23/156): python3-rbd-18.2.2-852.g234d5323.el9. 3.9 MB/s | 298 kB 00:00 2024-03-14T21:11:23.563 INFO:teuthology.orchestra.run.smithi097.stdout:(24/156): python3-rgw-18.2.2-852.g234d5323.el9. 1.4 MB/s | 99 kB 00:00 2024-03-14T21:11:23.589 INFO:teuthology.orchestra.run.smithi097.stdout:(25/156): rbd-fuse-18.2.2-852.g234d5323.el9.x86 1.6 MB/s | 87 kB 00:00 2024-03-14T21:11:23.672 INFO:teuthology.orchestra.run.smithi097.stdout:(26/156): rbd-nbd-18.2.2-852.g234d5323.el9.x86_ 2.0 MB/s | 172 kB 00:00 2024-03-14T21:11:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:(27/156): ceph-grafana-dashboards-18.2.2-852.g2 255 kB/s | 23 kB 00:00 2024-03-14T21:11:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:(28/156): rbd-mirror-18.2.2-852.g234d5323.el9.x 11 MB/s | 3.0 MB 00:00 2024-03-14T21:11:23.857 INFO:teuthology.orchestra.run.smithi097.stdout:(29/156): ceph-mgr-cephadm-18.2.2-852.g234d5323 1.5 MB/s | 138 kB 00:00 2024-03-14T21:11:24.074 INFO:teuthology.orchestra.run.smithi097.stdout:(30/156): ceph-mgr-dashboard-18.2.2-852.g234d53 7.2 MB/s | 1.7 MB 00:00 2024-03-14T21:11:24.150 INFO:teuthology.orchestra.run.smithi097.stdout:(31/156): ceph-mgr-modules-core-18.2.2-852.g234 3.2 MB/s | 245 kB 00:00 2024-03-14T21:11:24.217 INFO:teuthology.orchestra.run.smithi097.stdout:(32/156): ceph-mgr-rook-18.2.2-852.g234d5323.el 740 kB/s | 49 kB 00:00 2024-03-14T21:11:24.283 INFO:teuthology.orchestra.run.smithi097.stdout:(33/156): ceph-prometheus-alerts-18.2.2-852.g23 223 kB/s | 15 kB 00:00 2024-03-14T21:11:24.359 INFO:teuthology.orchestra.run.smithi097.stdout:(34/156): ceph-volume-18.2.2-852.g234d5323.el9. 3.4 MB/s | 264 kB 00:00 2024-03-14T21:11:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:(35/156): cephadm-18.2.2-852.g234d5323.el9.noar 2.9 MB/s | 224 kB 00:00 2024-03-14T21:11:24.445 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-14T21:11:24.493 INFO:teuthology.orchestra.run.smithi097.stdout:(36/156): cephfs-top-18.2.2-852.g234d5323.el9.n 310 kB/s | 18 kB 00:00 2024-03-14T21:11:24.593 INFO:teuthology.orchestra.run.smithi097.stdout:(37/156): ceph-mgr-diskprediction-local-18.2.2- 10 MB/s | 7.4 MB 00:00 2024-03-14T21:11:24.760 INFO:teuthology.orchestra.run.smithi097.stdout:(38/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-14T21:11:24.794 INFO:teuthology.orchestra.run.smithi097.stdout:(39/156): bison-3.7.4-5.el9.x86_64.rpm 5.1 MB/s | 920 kB 00:00 2024-03-14T21:11:24.819 INFO:teuthology.orchestra.run.smithi097.stdout:(40/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-14T21:11:24.852 INFO:teuthology.orchestra.run.smithi097.stdout:(41/156): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-14T21:11:24.869 INFO:teuthology.orchestra.run.smithi097.stdout:(42/156): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-14T21:11:24.953 INFO:teuthology.orchestra.run.smithi097.stdout:(43/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-14T21:11:24.970 INFO:teuthology.orchestra.run.smithi097.stdout:(44/156): flexiblas-openblas-openmp-3.0.4-8.el9 171 kB/s | 17 kB 00:00 2024-03-14T21:11:24.987 INFO:teuthology.orchestra.run.smithi097.stdout:(45/156): libaio-devel-0.3.111-13.el9.x86_64.rp 300 kB/s | 10 kB 00:00 2024-03-14T21:11:25.012 INFO:teuthology.orchestra.run.smithi097.stdout:(46/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-14T21:11:25.029 INFO:teuthology.orchestra.run.smithi097.stdout:(47/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-14T21:11:25.063 INFO:teuthology.orchestra.run.smithi097.stdout:(48/156): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-14T21:11:25.088 INFO:teuthology.orchestra.run.smithi097.stdout:(49/156): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2024-03-14T21:11:25.105 INFO:teuthology.orchestra.run.smithi097.stdout:(50/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-14T21:11:25.130 INFO:teuthology.orchestra.run.smithi097.stdout:(51/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-14T21:11:25.164 INFO:teuthology.orchestra.run.smithi097.stdout:(52/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-14T21:11:25.181 INFO:teuthology.orchestra.run.smithi097.stdout:(53/156): openblas-0.3.21-2.el9.x86_64.rpm 684 kB/s | 34 kB 00:00 2024-03-14T21:11:25.281 INFO:teuthology.orchestra.run.smithi097.stdout:(54/156): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-14T21:11:25.323 INFO:teuthology.orchestra.run.smithi097.stdout:(55/156): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-14T21:11:25.357 INFO:teuthology.orchestra.run.smithi097.stdout:(56/156): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-14T21:11:25.382 INFO:teuthology.orchestra.run.smithi097.stdout:(57/156): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-14T21:11:25.508 INFO:teuthology.orchestra.run.smithi097.stdout:(58/156): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-03-14T21:11:25.616 INFO:teuthology.orchestra.run.smithi097.stdout:(59/156): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-14T21:11:25.641 INFO:teuthology.orchestra.run.smithi097.stdout:(60/156): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-03-14T21:11:25.666 INFO:teuthology.orchestra.run.smithi097.stdout:(61/156): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-14T21:11:25.692 INFO:teuthology.orchestra.run.smithi097.stdout:(62/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-14T21:11:25.717 INFO:teuthology.orchestra.run.smithi097.stdout:(63/156): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-14T21:11:26.093 INFO:teuthology.orchestra.run.smithi097.stdout:(64/156): ceph-test-18.2.2-852.g234d5323.el9.x8 10 MB/s | 40 MB 00:03 2024-03-14T21:11:26.168 INFO:teuthology.orchestra.run.smithi097.stdout:(65/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 4.8 MB 00:00 2024-03-14T21:11:26.202 INFO:teuthology.orchestra.run.smithi097.stdout:(66/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 451 kB/s | 217 kB 00:00 2024-03-14T21:11:26.227 INFO:teuthology.orchestra.run.smithi097.stdout:(67/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-14T21:11:26.252 INFO:teuthology.orchestra.run.smithi097.stdout:(68/156): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-03-14T21:11:26.277 INFO:teuthology.orchestra.run.smithi097.stdout:(69/156): python3-prettytable-0.7.2-27.el9.noar 551 kB/s | 41 kB 00:00 2024-03-14T21:11:26.303 INFO:teuthology.orchestra.run.smithi097.stdout:(70/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-14T21:11:26.328 INFO:teuthology.orchestra.run.smithi097.stdout:(71/156): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-03-14T21:11:26.353 INFO:teuthology.orchestra.run.smithi097.stdout:(72/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-14T21:11:26.370 INFO:teuthology.orchestra.run.smithi097.stdout:(73/156): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 43 kB 00:00 2024-03-14T21:11:26.595 INFO:teuthology.orchestra.run.smithi097.stdout:(74/156): python3-toml-0.10.2-6.el9.noarch.rpm 174 kB/s | 39 kB 00:00 2024-03-14T21:11:26.624 INFO:teuthology.orchestra.run.smithi097.stdout:(75/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-03-14T21:11:26.782 INFO:teuthology.orchestra.run.smithi097.stdout:(76/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-14T21:11:26.807 INFO:teuthology.orchestra.run.smithi097.stdout:(77/156): uuid-1.6.2-55.el9.x86_64.rpm 266 kB/s | 56 kB 00:00 2024-03-14T21:11:26.833 INFO:teuthology.orchestra.run.smithi097.stdout:(78/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 300 kB/s | 63 kB 00:00 2024-03-14T21:11:26.858 INFO:teuthology.orchestra.run.smithi097.stdout:(79/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-03-14T21:11:26.883 INFO:teuthology.orchestra.run.smithi097.stdout:(80/156): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-14T21:11:26.908 INFO:teuthology.orchestra.run.smithi097.stdout:(81/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-14T21:11:26.934 INFO:teuthology.orchestra.run.smithi097.stdout:(82/156): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-14T21:11:26.976 INFO:teuthology.orchestra.run.smithi097.stdout:(83/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-14T21:11:27.001 INFO:teuthology.orchestra.run.smithi097.stdout:(84/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-14T21:11:27.026 INFO:teuthology.orchestra.run.smithi097.stdout:(85/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-14T21:11:27.197 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T21:11:27.199 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T21:11:27.221 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-14T21:11:27.222 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-14T21:11:27.223 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-14T21:11:27.225 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-14T21:11:27.227 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-14T21:11:27.231 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-14T21:11:27.233 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-14T21:11:27.317 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================== 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================== 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:18.2.2-852.g234d5323.el9 ceph 6.5 k 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:18.2.2-852.g234d5323.el9 ceph 5.2 M 2024-03-14T21:11:27.332 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:18.2.2-852.g234d5323.el9 ceph 843 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-852.g234d5323.el9 ceph 142 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:18.2.2-852.g234d5323.el9 ceph 1.4 M 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 138 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 1.7 M 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 7.4 M 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 49 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:18.2.2-852.g234d5323.el9 ceph 7.7 M 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:18.2.2-852.g234d5323.el9 ceph 40 M 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 264 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 224 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:18.2.2-852.g234d5323.el9 ceph 222 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 18 k 2024-03-14T21:11:27.333 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:18.2.2-852.g234d5323.el9 ceph 31 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 659 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:18.2.2-852.g234d5323.el9 ceph 127 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-14T21:11:27.334 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:18.2.2-852.g234d5323.el9 ceph 161 k 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:18.2.2-852.g234d5323.el9 ceph 321 k 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:18.2.2-852.g234d5323.el9 ceph 298 k 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:18.2.2-852.g234d5323.el9 ceph 99 k 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:18.2.2-852.g234d5323.el9 ceph 87 k 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.0 M 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:18.2.2-852.g234d5323.el9 ceph 172 k 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-14T21:11:27.335 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-14T21:11:27.336 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.3 M 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.0 M 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-14T21:11:27.337 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:18.2.2-852.g234d5323.el9 ceph 18 M 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 23 k 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:18.2.2-852.g234d5323.el9 ceph 2.1 M 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 245 k 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:18.2.2-852.g234d5323.el9 ceph 4.5 M 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:18.2.2-852.g234d5323.el9 ceph 18 M 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 15 k 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:18.2.2-852.g234d5323.el9 ceph 25 k 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-14T21:11:27.338 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:18.2.2-852.g234d5323.el9 ceph 166 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-14T21:11:27.339 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:18.2.2-852.g234d5323.el9 ceph 474 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 4.5 M 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-14T21:11:27.340 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:18.2.2-852.g234d5323.el9 ceph 45 k 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:18.2.2-852.g234d5323.el9 ceph 127 k 2024-03-14T21:11:27.341 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-14T21:11:27.342 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-14T21:11:27.343 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-14T21:11:27.344 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-14T21:11:27.345 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-03-14T21:11:27.346 INFO:teuthology.orchestra.run.smithi163.stdout:=========================================================================================== 2024-03-14T21:11:27.347 INFO:teuthology.orchestra.run.smithi163.stdout:Install 132 Packages 2024-03-14T21:11:27.347 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 24 Packages 2024-03-14T21:11:27.347 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:11:27.348 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 205 M 2024-03-14T21:11:27.348 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-03-14T21:11:27.852 INFO:teuthology.orchestra.run.smithi097.stdout:(86/156): libquadmath-11.4.1-3.el9.x86_64.rpm 237 kB/s | 195 kB 00:00 2024-03-14T21:11:27.885 INFO:teuthology.orchestra.run.smithi097.stdout:(87/156): jq-1.6-16.el9.x86_64.rpm 212 kB/s | 187 kB 00:00 2024-03-14T21:11:27.977 INFO:teuthology.orchestra.run.smithi097.stdout:(88/156): elfutils-libelf-devel-0.190-2.el9.x86 235 kB/s | 21 kB 00:00 2024-03-14T21:11:28.027 INFO:teuthology.orchestra.run.smithi097.stdout:(89/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-03-14T21:11:28.245 INFO:teuthology.orchestra.run.smithi097.stdout:(90/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 19 MB/s | 4.1 MB 00:00 2024-03-14T21:11:28.280 INFO:teuthology.orchestra.run.smithi163.stdout:(1/156): ceph-18.2.2-852.g234d5323.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2024-03-14T21:11:28.303 INFO:teuthology.orchestra.run.smithi097.stdout:(91/156): python3-devel-3.9.17-2.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-03-14T21:11:28.353 INFO:teuthology.orchestra.run.smithi097.stdout:(92/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-03-14T21:11:28.395 INFO:teuthology.orchestra.run.smithi097.stdout:(93/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-03-14T21:11:28.479 INFO:teuthology.orchestra.run.smithi097.stdout:(94/156): fmt-8.1.1-5.el9.x86_64.rpm 1.3 MB/s | 111 kB 00:00 2024-03-14T21:11:28.538 INFO:teuthology.orchestra.run.smithi097.stdout:(95/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 5.2 MB/s | 309 kB 00:00 2024-03-14T21:11:28.638 INFO:teuthology.orchestra.run.smithi097.stdout:(96/156): python3-requests-2.25.1-8.el9.noarch. 159 kB/s | 125 kB 00:00 2024-03-14T21:11:28.705 INFO:teuthology.orchestra.run.smithi097.stdout:(97/156): libarrow-9.0.0-10.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-03-14T21:11:28.739 INFO:teuthology.orchestra.run.smithi097.stdout:(98/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-03-14T21:11:28.747 INFO:teuthology.orchestra.run.smithi163.stdout:(2/156): ceph-fuse-18.2.2-852.g234d5323.el9.x86 1.8 MB/s | 843 kB 00:00 2024-03-14T21:11:28.764 INFO:teuthology.orchestra.run.smithi097.stdout:(99/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-03-14T21:11:28.797 INFO:teuthology.orchestra.run.smithi097.stdout:(100/156): libgfortran-11.4.1-3.el9.x86_64.rpm 447 kB/s | 803 kB 00:01 2024-03-14T21:11:28.823 INFO:teuthology.orchestra.run.smithi097.stdout:(101/156): libunwind-1.6.2-1.el9.x86_64.rpm 804 kB/s | 67 kB 00:00 2024-03-14T21:11:28.830 INFO:teuthology.orchestra.run.smithi163.stdout:(3/156): ceph-immutable-object-cache-18.2.2-852 1.7 MB/s | 142 kB 00:00 2024-03-14T21:11:28.865 INFO:teuthology.orchestra.run.smithi097.stdout:(102/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-14T21:11:28.907 INFO:teuthology.orchestra.run.smithi097.stdout:(103/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.7 MB/s | 838 kB 00:00 2024-03-14T21:11:28.932 INFO:teuthology.orchestra.run.smithi097.stdout:(104/156): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-03-14T21:11:28.957 INFO:teuthology.orchestra.run.smithi097.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-03-14T21:11:28.982 INFO:teuthology.orchestra.run.smithi097.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-03-14T21:11:29.008 INFO:teuthology.orchestra.run.smithi097.stdout:(107/156): python3-asyncssh-2.13.2-4.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-03-14T21:11:29.033 INFO:teuthology.orchestra.run.smithi097.stdout:(108/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-03-14T21:11:29.058 INFO:teuthology.orchestra.run.smithi097.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-03-14T21:11:29.084 INFO:teuthology.orchestra.run.smithi097.stdout:(110/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-03-14T21:11:29.109 INFO:teuthology.orchestra.run.smithi097.stdout:(111/156): python3-google-auth-2.28.0-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-14T21:11:29.134 INFO:teuthology.orchestra.run.smithi097.stdout:(112/156): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-03-14T21:11:29.159 INFO:teuthology.orchestra.run.smithi097.stdout:(113/156): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-03-14T21:11:29.185 INFO:teuthology.orchestra.run.smithi097.stdout:(114/156): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-03-14T21:11:29.210 INFO:teuthology.orchestra.run.smithi097.stdout:(115/156): python3-logutils-0.3.5-21.el9.noarch 923 kB/s | 46 kB 00:00 2024-03-14T21:11:29.236 INFO:teuthology.orchestra.run.smithi097.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-14T21:11:29.261 INFO:teuthology.orchestra.run.smithi097.stdout:(117/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-14T21:11:29.286 INFO:teuthology.orchestra.run.smithi097.stdout:(118/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-14T21:11:29.311 INFO:teuthology.orchestra.run.smithi097.stdout:(119/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-14T21:11:29.337 INFO:teuthology.orchestra.run.smithi097.stdout:(120/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-03-14T21:11:29.362 INFO:teuthology.orchestra.run.smithi097.stdout:(121/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-03-14T21:11:29.387 INFO:teuthology.orchestra.run.smithi097.stdout:(122/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-14T21:11:29.412 INFO:teuthology.orchestra.run.smithi097.stdout:(123/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-14T21:11:29.438 INFO:teuthology.orchestra.run.smithi097.stdout:(124/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-03-14T21:11:29.447 INFO:teuthology.orchestra.run.smithi163.stdout:(4/156): ceph-base-18.2.2-852.g234d5323.el9.x86 3.5 MB/s | 5.2 MB 00:01 2024-03-14T21:11:29.463 INFO:teuthology.orchestra.run.smithi097.stdout:(125/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-14T21:11:29.497 INFO:teuthology.orchestra.run.smithi097.stdout:(126/156): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-03-14T21:11:29.522 INFO:teuthology.orchestra.run.smithi097.stdout:(127/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-14T21:11:29.522 INFO:teuthology.orchestra.run.smithi163.stdout:(5/156): ceph-mds-18.2.2-852.g234d5323.el9.x86_ 3.1 MB/s | 2.1 MB 00:00 2024-03-14T21:11:29.547 INFO:teuthology.orchestra.run.smithi097.stdout:(128/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-14T21:11:29.589 INFO:teuthology.orchestra.run.smithi097.stdout:(129/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-03-14T21:11:29.615 INFO:teuthology.orchestra.run.smithi097.stdout:(130/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-14T21:11:29.648 INFO:teuthology.orchestra.run.smithi097.stdout:(131/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-03-14T21:11:29.656 INFO:teuthology.orchestra.run.smithi163.stdout:(6/156): ceph-mgr-18.2.2-852.g234d5323.el9.x86_ 7.0 MB/s | 1.4 MB 00:00 2024-03-14T21:11:29.690 INFO:teuthology.orchestra.run.smithi097.stdout:(132/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-14T21:11:29.749 INFO:teuthology.orchestra.run.smithi097.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-14T21:11:30.274 INFO:teuthology.orchestra.run.smithi097.stdout:(134/156): librados2-18.2.2-852.g234d5323.el9.x 5.0 MB/s | 3.3 MB 00:00 2024-03-14T21:11:30.333 INFO:teuthology.orchestra.run.smithi097.stdout:(135/156): librbd1-18.2.2-852.g234d5323.el9.x86 4.4 MB/s | 3.0 MB 00:00 2024-03-14T21:11:30.498 INFO:teuthology.orchestra.run.smithi163.stdout:(7/156): ceph-mon-18.2.2-852.g234d5323.el9.x86_ 4.6 MB/s | 4.5 MB 00:00 2024-03-14T21:11:31.000 INFO:teuthology.orchestra.run.smithi097.stdout:(136/156): NetworkManager-team-1.46.0-1.el9.x86 63 kB/s | 42 kB 00:00 2024-03-14T21:11:31.490 INFO:teuthology.orchestra.run.smithi163.stdout:(8/156): ceph-osd-18.2.2-852.g234d5323.el9.x86_ 9.6 MB/s | 18 MB 00:01 2024-03-14T21:11:31.557 INFO:teuthology.orchestra.run.smithi163.stdout:(9/156): ceph-selinux-18.2.2-852.g234d5323.el9. 387 kB/s | 25 kB 00:00 2024-03-14T21:11:31.799 INFO:teuthology.orchestra.run.smithi163.stdout:(10/156): ceph-radosgw-18.2.2-852.g234d5323.el9 5.9 MB/s | 7.7 MB 00:01 2024-03-14T21:11:31.942 INFO:teuthology.orchestra.run.smithi097.stdout:(137/156): NetworkManager-tui-1.46.0-1.el9.x86_ 264 kB/s | 248 kB 00:00 2024-03-14T21:11:32.049 INFO:teuthology.orchestra.run.smithi163.stdout:(11/156): ceph-common-18.2.2-852.g234d5323.el9. 4.5 MB/s | 18 MB 00:04 2024-03-14T21:11:32.082 INFO:teuthology.orchestra.run.smithi163.stdout:(12/156): cephfs-mirror-18.2.2-852.g234d5323.el 785 kB/s | 222 kB 00:00 2024-03-14T21:11:32.133 INFO:teuthology.orchestra.run.smithi163.stdout:(13/156): libcephfs-devel-18.2.2-852.g234d5323. 377 kB/s | 31 kB 00:00 2024-03-14T21:11:32.216 INFO:teuthology.orchestra.run.smithi163.stdout:(14/156): libcephsqlite-18.2.2-852.g234d5323.el 2.0 MB/s | 166 kB 00:00 2024-03-14T21:11:32.258 INFO:teuthology.orchestra.run.smithi163.stdout:(15/156): libcephfs2-18.2.2-852.g234d5323.el9.x 3.7 MB/s | 659 kB 00:00 2024-03-14T21:11:32.300 INFO:teuthology.orchestra.run.smithi163.stdout:(16/156): librados-devel-18.2.2-852.g234d5323.e 1.5 MB/s | 127 kB 00:00 2024-03-14T21:11:32.392 INFO:teuthology.orchestra.run.smithi163.stdout:(17/156): libradosstriper1-18.2.2-852.g234d5323 3.5 MB/s | 474 kB 00:00 2024-03-14T21:11:32.513 INFO:teuthology.orchestra.run.smithi163.stdout:(18/156): python3-ceph-argparse-18.2.2-852.g234 376 kB/s | 45 kB 00:00 2024-03-14T21:11:32.576 INFO:teuthology.orchestra.run.smithi163.stdout:(19/156): python3-ceph-common-18.2.2-852.g234d5 2.0 MB/s | 127 kB 00:00 2024-03-14T21:11:32.651 INFO:teuthology.orchestra.run.smithi163.stdout:(20/156): python3-cephfs-18.2.2-852.g234d5323.e 2.1 MB/s | 161 kB 00:00 2024-03-14T21:11:32.658 INFO:teuthology.orchestra.run.smithi097.stdout:(138/156): acl-2.3.1-4.el9.x86_64.rpm 100 kB/s | 71 kB 00:00 2024-03-14T21:11:32.743 INFO:teuthology.orchestra.run.smithi163.stdout:(21/156): python3-rados-18.2.2-852.g234d5323.el 3.5 MB/s | 321 kB 00:00 2024-03-14T21:11:32.835 INFO:teuthology.orchestra.run.smithi163.stdout:(22/156): python3-rbd-18.2.2-852.g234d5323.el9. 3.2 MB/s | 298 kB 00:00 2024-03-14T21:11:33.002 INFO:teuthology.orchestra.run.smithi163.stdout:(23/156): python3-rgw-18.2.2-852.g234d5323.el9. 600 kB/s | 99 kB 00:00 2024-03-14T21:11:33.127 INFO:teuthology.orchestra.run.smithi163.stdout:(24/156): librgw2-18.2.2-852.g234d5323.el9.x86_ 5.4 MB/s | 4.5 MB 00:00 2024-03-14T21:11:33.153 INFO:teuthology.orchestra.run.smithi163.stdout:(25/156): rbd-fuse-18.2.2-852.g234d5323.el9.x86 581 kB/s | 87 kB 00:00 2024-03-14T21:11:33.220 INFO:teuthology.orchestra.run.smithi163.stdout:(26/156): rbd-nbd-18.2.2-852.g234d5323.el9.x86_ 2.6 MB/s | 172 kB 00:00 2024-03-14T21:11:33.251 INFO:teuthology.orchestra.run.smithi097.stdout:(139/156): NetworkManager-libnm-1.46.0-1.el9.x8 625 kB/s | 1.8 MB 00:02 2024-03-14T21:11:33.300 INFO:teuthology.orchestra.run.smithi097.stdout:(140/156): elfutils-debuginfod-client-0.190-2.e 57 kB/s | 36 kB 00:00 2024-03-14T21:11:33.303 INFO:teuthology.orchestra.run.smithi163.stdout:(27/156): ceph-grafana-dashboards-18.2.2-852.g2 284 kB/s | 23 kB 00:00 2024-03-14T21:11:33.387 INFO:teuthology.orchestra.run.smithi163.stdout:(28/156): ceph-mgr-cephadm-18.2.2-852.g234d5323 1.6 MB/s | 138 kB 00:00 2024-03-14T21:11:33.476 INFO:teuthology.orchestra.run.smithi097.stdout:(141/156): NetworkManager-1.46.0-1.el9.x86_64.r 629 kB/s | 2.3 MB 00:03 2024-03-14T21:11:33.604 INFO:teuthology.orchestra.run.smithi163.stdout:(29/156): rbd-mirror-18.2.2-852.g234d5323.el9.x 6.3 MB/s | 3.0 MB 00:00 2024-03-14T21:11:33.737 INFO:teuthology.orchestra.run.smithi163.stdout:(30/156): ceph-mgr-dashboard-18.2.2-852.g234d53 5.0 MB/s | 1.7 MB 00:00 2024-03-14T21:11:33.821 INFO:teuthology.orchestra.run.smithi163.stdout:(31/156): ceph-mgr-modules-core-18.2.2-852.g234 2.9 MB/s | 245 kB 00:00 2024-03-14T21:11:33.921 INFO:teuthology.orchestra.run.smithi163.stdout:(32/156): ceph-mgr-rook-18.2.2-852.g234d5323.el 499 kB/s | 49 kB 00:00 2024-03-14T21:11:34.005 INFO:teuthology.orchestra.run.smithi163.stdout:(33/156): ceph-prometheus-alerts-18.2.2-852.g23 181 kB/s | 15 kB 00:00 2024-03-14T21:11:34.097 INFO:teuthology.orchestra.run.smithi163.stdout:(34/156): ceph-volume-18.2.2-852.g234d5323.el9. 2.8 MB/s | 264 kB 00:00 2024-03-14T21:11:34.101 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 871 kB/s | 19 MB 00:22 2024-03-14T21:11:34.143 INFO:teuthology.orchestra.run.smithi097.stdout:(142/156): libacl-2.3.1-4.el9.x86_64.rpm 36 kB/s | 23 kB 00:00 2024-03-14T21:11:34.168 INFO:teuthology.orchestra.run.smithi097.stdout:(143/156): elfutils-libelf-0.190-2.el9.x86_64.r 210 kB/s | 192 kB 00:00 2024-03-14T21:11:34.189 INFO:teuthology.orchestra.run.smithi163.stdout:(35/156): cephadm-18.2.2-852.g234d5323.el9.noar 2.4 MB/s | 224 kB 00:00 2024-03-14T21:11:34.272 INFO:teuthology.orchestra.run.smithi163.stdout:(36/156): cephfs-top-18.2.2-852.g234d5323.el9.n 220 kB/s | 18 kB 00:00 2024-03-14T21:11:34.294 INFO:teuthology.orchestra.run.smithi097.stdout:(144/156): elfutils-libs-0.190-2.el9.x86_64.rpm 258 kB/s | 256 kB 00:00 2024-03-14T21:11:34.456 INFO:teuthology.orchestra.run.smithi163.stdout:(37/156): automake-1.16.2-8.el9.noarch.rpm 3.6 MB/s | 662 kB 00:00 2024-03-14T21:11:34.551 INFO:teuthology.orchestra.run.smithi163.stdout:(38/156): bison-3.7.4-5.el9.x86_64.rpm 9.6 MB/s | 920 kB 00:00 2024-03-14T21:11:34.598 INFO:teuthology.orchestra.run.smithi163.stdout:(39/156): boost-program-options-1.75.0-8.el9.x8 2.3 MB/s | 106 kB 00:00 2024-03-14T21:11:34.640 INFO:teuthology.orchestra.run.smithi163.stdout:(40/156): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 307 kB 00:00 2024-03-14T21:11:34.682 INFO:teuthology.orchestra.run.smithi163.stdout:(41/156): flexiblas-3.0.4-8.el9.x86_64.rpm 782 kB/s | 32 kB 00:00 2024-03-14T21:11:34.824 INFO:teuthology.orchestra.run.smithi163.stdout:(42/156): ceph-mgr-diskprediction-local-18.2.2- 6.1 MB/s | 7.4 MB 00:01 2024-03-14T21:11:34.894 INFO:teuthology.orchestra.run.smithi097.stdout:(145/156): libblkid-2.37.4-18.el9.x86_64.rpm 144 kB/s | 108 kB 00:00 2024-03-14T21:11:34.916 INFO:teuthology.orchestra.run.smithi163.stdout:(43/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-03-14T21:11:34.941 INFO:teuthology.orchestra.run.smithi163.stdout:(44/156): flexiblas-openblas-openmp-3.0.4-8.el9 166 kB/s | 17 kB 00:00 2024-03-14T21:11:34.974 INFO:teuthology.orchestra.run.smithi163.stdout:(45/156): libaio-devel-0.3.111-13.el9.x86_64.rp 312 kB/s | 10 kB 00:00 2024-03-14T21:11:34.994 INFO:teuthology.orchestra.run.smithi097.stdout:(146/156): libfdisk-2.37.4-18.el9.x86_64.rpm 187 kB/s | 155 kB 00:00 2024-03-14T21:11:35.024 INFO:teuthology.orchestra.run.smithi163.stdout:(46/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-14T21:11:35.050 INFO:teuthology.orchestra.run.smithi163.stdout:(47/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 44 kB 00:00 2024-03-14T21:11:35.053 INFO:teuthology.orchestra.run.smithi097.stdout:(147/156): libmount-2.37.4-18.el9.x86_64.rpm 179 kB/s | 136 kB 00:00 2024-03-14T21:11:35.100 INFO:teuthology.orchestra.run.smithi163.stdout:(48/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-03-14T21:11:35.142 INFO:teuthology.orchestra.run.smithi163.stdout:(49/156): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-14T21:11:35.167 INFO:teuthology.orchestra.run.smithi163.stdout:(50/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 766 kB/s | 46 kB 00:00 2024-03-14T21:11:35.201 INFO:teuthology.orchestra.run.smithi163.stdout:(51/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-14T21:11:35.235 INFO:teuthology.orchestra.run.smithi163.stdout:(52/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-14T21:11:35.260 INFO:teuthology.orchestra.run.smithi163.stdout:(53/156): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 34 kB 00:00 2024-03-14T21:11:35.645 INFO:teuthology.orchestra.run.smithi097.stdout:(148/156): libuuid-2.37.4-18.el9.x86_64.rpm 44 kB/s | 29 kB 00:00 2024-03-14T21:11:35.670 INFO:teuthology.orchestra.run.smithi097.stdout:(149/156): libsmartcols-2.37.4-18.el9.x86_64.rp 82 kB/s | 64 kB 00:00 2024-03-14T21:11:35.761 INFO:teuthology.orchestra.run.smithi163.stdout:(54/156): ceph-test-18.2.2-852.g234d5323.el9.x8 9.4 MB/s | 40 MB 00:04 2024-03-14T21:11:35.894 INFO:teuthology.orchestra.run.smithi163.stdout:(55/156): openblas-openmp-0.3.21-2.el9.x86_64.r 7.2 MB/s | 4.7 MB 00:00 2024-03-14T21:11:35.952 INFO:teuthology.orchestra.run.smithi163.stdout:(56/156): postgresql-13.11-1.el9.x86_64.rpm 2.2 MB/s | 1.5 MB 00:00 2024-03-14T21:11:35.986 INFO:teuthology.orchestra.run.smithi163.stdout:(57/156): postgresql-contrib-13.11-1.el9.x86_64 3.8 MB/s | 811 kB 00:00 2024-03-14T21:11:36.011 INFO:teuthology.orchestra.run.smithi163.stdout:(58/156): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-14T21:11:36.153 INFO:teuthology.orchestra.run.smithi163.stdout:(59/156): python3-jsonpatch-1.21-16.el9.noarch. 173 kB/s | 24 kB 00:00 2024-03-14T21:11:36.262 INFO:teuthology.orchestra.run.smithi163.stdout:(60/156): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-14T21:11:36.379 INFO:teuthology.orchestra.run.smithi163.stdout:(61/156): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-03-14T21:11:36.395 INFO:teuthology.orchestra.run.smithi163.stdout:(62/156): python3-jsonpointer-2.0-4.el9.noarch. 76 kB/s | 18 kB 00:00 2024-03-14T21:11:36.420 INFO:teuthology.orchestra.run.smithi163.stdout:(63/156): python3-mako-1.1.4-6.el9.noarch.rpm 930 kB/s | 144 kB 00:00 2024-03-14T21:11:36.454 INFO:teuthology.orchestra.run.smithi163.stdout:(64/156): python3-markupsafe-1.1.1-12.el9.x86_6 561 kB/s | 32 kB 00:00 2024-03-14T21:11:36.571 INFO:teuthology.orchestra.run.smithi163.stdout:(65/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 4.8 MB 00:00 2024-03-14T21:11:36.597 INFO:teuthology.orchestra.run.smithi163.stdout:(66/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-14T21:11:36.621 INFO:teuthology.orchestra.run.smithi163.stdout:(67/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-14T21:11:36.663 INFO:teuthology.orchestra.run.smithi163.stdout:(68/156): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-14T21:11:36.680 INFO:teuthology.orchestra.run.smithi163.stdout:(69/156): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 41 kB 00:00 2024-03-14T21:11:36.722 INFO:teuthology.orchestra.run.smithi163.stdout:(70/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2024-03-14T21:11:36.764 INFO:teuthology.orchestra.run.smithi163.stdout:(71/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 47 kB 00:00 2024-03-14T21:11:36.798 INFO:teuthology.orchestra.run.smithi163.stdout:(72/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-14T21:11:36.814 INFO:teuthology.orchestra.run.smithi163.stdout:(73/156): python3-requests-oauthlib-1.3.0-12.el 474 kB/s | 43 kB 00:00 2024-03-14T21:11:37.248 INFO:teuthology.orchestra.run.smithi163.stdout:(74/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-03-14T21:11:37.273 INFO:teuthology.orchestra.run.smithi163.stdout:(75/156): python3-toml-0.10.2-6.el9.noarch.rpm 87 kB/s | 39 kB 00:00 2024-03-14T21:11:37.298 INFO:teuthology.orchestra.run.smithi163.stdout:(76/156): socat-1.7.4.1-5.el9.x86_64.rpm 640 kB/s | 300 kB 00:00 2024-03-14T21:11:37.315 INFO:teuthology.orchestra.run.smithi163.stdout:(77/156): uuid-1.6.2-55.el9.x86_64.rpm 893 kB/s | 56 kB 00:00 2024-03-14T21:11:37.332 INFO:teuthology.orchestra.run.smithi163.stdout:(78/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-14T21:11:37.349 INFO:teuthology.orchestra.run.smithi163.stdout:(79/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-14T21:11:37.374 INFO:teuthology.orchestra.run.smithi163.stdout:(80/156): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-14T21:11:37.408 INFO:teuthology.orchestra.run.smithi163.stdout:(81/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-14T21:11:37.433 INFO:teuthology.orchestra.run.smithi163.stdout:(82/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-14T21:11:37.484 INFO:teuthology.orchestra.run.smithi163.stdout:(83/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-14T21:11:37.487 INFO:teuthology.orchestra.run.smithi097.stdout:(150/156): openssl-3.0.7-27.el9.x86_64.rpm 509 kB/s | 1.2 MB 00:02 2024-03-14T21:11:37.509 INFO:teuthology.orchestra.run.smithi163.stdout:(84/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-14T21:11:37.542 INFO:teuthology.orchestra.run.smithi163.stdout:(85/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 187 kB 00:00 2024-03-14T21:11:37.668 INFO:teuthology.orchestra.run.smithi163.stdout:(86/156): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-14T21:11:37.709 INFO:teuthology.orchestra.run.smithi163.stdout:(87/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-14T21:11:37.760 INFO:teuthology.orchestra.run.smithi163.stdout:(88/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-03-14T21:11:37.802 INFO:teuthology.orchestra.run.smithi163.stdout:(89/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-03-14T21:11:37.994 INFO:teuthology.orchestra.run.smithi163.stdout:(90/156): elfutils-libelf-devel-0.190-2.el9.x86 76 kB/s | 21 kB 00:00 2024-03-14T21:11:38.035 INFO:teuthology.orchestra.run.smithi163.stdout:(91/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 93 kB/s | 25 kB 00:00 2024-03-14T21:11:38.176 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-14T21:11:38.361 INFO:teuthology.orchestra.run.smithi163.stdout:(92/156): python3-devel-3.9.17-2.el9.x86_64.rpm 670 kB/s | 245 kB 00:00 2024-03-14T21:11:38.478 INFO:teuthology.orchestra.run.smithi163.stdout:(93/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 808 kB/s | 94 kB 00:00 2024-03-14T21:11:38.511 INFO:teuthology.orchestra.run.smithi163.stdout:(94/156): python3-jinja2-2.11.3-5.el9.noarch.rp 522 kB/s | 248 kB 00:00 2024-03-14T21:11:38.953 INFO:teuthology.orchestra.run.smithi163.stdout:(95/156): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 4.1 MB 00:01 2024-03-14T21:11:39.079 INFO:teuthology.orchestra.run.smithi097.stdout:(151/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 643 kB/s | 2.2 MB 00:03 2024-03-14T21:11:39.162 INFO:teuthology.orchestra.run.smithi163.stdout:(96/156): fmt-8.1.1-5.el9.x86_64.rpm 162 kB/s | 111 kB 00:00 2024-03-14T21:11:39.246 INFO:teuthology.orchestra.run.smithi097.stdout:(152/156): util-linux-2.37.4-18.el9.x86_64.rpm 654 kB/s | 2.3 MB 00:03 2024-03-14T21:11:39.370 INFO:teuthology.orchestra.run.smithi163.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 125 kB/s | 26 kB 00:00 2024-03-14T21:11:39.587 INFO:teuthology.orchestra.run.smithi163.stdout:(98/156): liboath-2.6.7-2.el9.x86_64.rpm 225 kB/s | 49 kB 00:00 2024-03-14T21:11:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 15 kB 00:00 2024-03-14T21:11:39.887 INFO:teuthology.orchestra.run.smithi163.stdout:(99/156): libunwind-1.6.2-1.el9.x86_64.rpm 225 kB/s | 67 kB 00:00 2024-03-14T21:11:39.921 INFO:teuthology.orchestra.run.smithi163.stdout:(100/156): gperftools-libs-2.9.1-2.el9.x86_64.r 219 kB/s | 309 kB 00:01 2024-03-14T21:11:40.379 INFO:teuthology.orchestra.run.smithi097.stdout:(153/156): util-linux-core-2.37.4-18.el9.x86_64 161 kB/s | 465 kB 00:02 2024-03-14T21:11:40.463 INFO:teuthology.orchestra.run.smithi097.stdout:(154/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 894 kB/s | 74 kB 00:00 2024-03-14T21:11:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:(155/156): util-linux-user-2.37.4-18.el9.x86_64 14 kB/s | 31 kB 00:02 2024-03-14T21:11:41.312 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-14T21:11:41.704 INFO:teuthology.orchestra.run.smithi163.stdout:(101/156): python3-asyncssh-2.13.2-4.el9.noarch 307 kB/s | 548 kB 00:01 2024-03-14T21:11:41.938 INFO:teuthology.orchestra.run.smithi163.stdout:(102/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 187 kB/s | 43 kB 00:00 2024-03-14T21:11:42.056 INFO:teuthology.orchestra.run.smithi163.stdout:(103/156): python3-cachetools-4.2.4-1.el9.noarc 278 kB/s | 32 kB 00:00 2024-03-14T21:11:42.163 INFO:teuthology.orchestra.run.smithi163.stdout:(104/156): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-03-14T21:11:42.305 INFO:teuthology.orchestra.run.smithi163.stdout:(105/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 347 kB/s | 838 kB 00:02 2024-03-14T21:11:42.689 INFO:teuthology.orchestra.run.smithi163.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 327 kB/s | 172 kB 00:00 2024-03-14T21:11:43.331 INFO:teuthology.orchestra.run.smithi163.stdout:(107/156): python3-google-auth-2.28.0-1.el9.noa 316 kB/s | 203 kB 00:00 2024-03-14T21:11:43.365 INFO:teuthology.orchestra.run.smithi163.stdout:(108/156): python3-cherrypy-18.6.1-2.el9.noarch 339 kB/s | 358 kB 00:01 2024-03-14T21:11:43.440 INFO:teuthology.orchestra.run.smithi163.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-03-14T21:11:43.482 INFO:teuthology.orchestra.run.smithi163.stdout:(110/156): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-03-14T21:11:43.549 INFO:teuthology.orchestra.run.smithi163.stdout:(111/156): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-03-14T21:11:43.571 INFO:teuthology.orchestra.run.smithi097.stdout:(156/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 254 kB/s | 1.1 MB 00:04 2024-03-14T21:11:43.577 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-14T21:11:43.577 INFO:teuthology.orchestra.run.smithi097.stdout:Total 8.7 MB/s | 205 MB 00:23 2024-03-14T21:11:43.590 INFO:teuthology.orchestra.run.smithi163.stdout:(112/156): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-03-14T21:11:43.658 INFO:teuthology.orchestra.run.smithi163.stdout:(113/156): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-03-14T21:11:43.882 INFO:teuthology.orchestra.run.smithi163.stdout:(114/156): python3-logutils-0.3.5-21.el9.noarch 207 kB/s | 46 kB 00:00 2024-03-14T21:11:44.011 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-14T21:11:44.092 INFO:teuthology.orchestra.run.smithi163.stdout:(115/156): python3-more-itertools-8.12.0-2.el9. 379 kB/s | 79 kB 00:00 2024-03-14T21:11:44.276 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-14T21:11:44.277 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-14T21:11:44.300 INFO:teuthology.orchestra.run.smithi163.stdout:(116/156): python3-natsort-7.1.1-5.el9.noarch.r 278 kB/s | 58 kB 00:00 2024-03-14T21:11:45.009 INFO:teuthology.orchestra.run.smithi163.stdout:(117/156): python3-pecan-1.4.2-3.el9.noarch.rpm 385 kB/s | 272 kB 00:00 2024-03-14T21:11:45.142 INFO:teuthology.orchestra.run.smithi163.stdout:(118/156): python3-portend-3.1.0-2.el9.noarch.r 124 kB/s | 16 kB 00:00 2024-03-14T21:11:45.492 INFO:teuthology.orchestra.run.smithi163.stdout:(119/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 258 kB/s | 90 kB 00:00 2024-03-14T21:11:45.618 INFO:teuthology.orchestra.run.smithi163.stdout:(120/156): python3-repoze-lru-0.7-16.el9.noarch 248 kB/s | 31 kB 00:00 2024-03-14T21:11:45.840 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-14T21:11:45.840 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-14T21:11:46.110 INFO:teuthology.orchestra.run.smithi163.stdout:(121/156): python3-routes-2.5.1-5.el9.noarch.rp 384 kB/s | 188 kB 00:00 2024-03-14T21:11:46.143 INFO:teuthology.orchestra.run.smithi163.stdout:(122/156): python3-kubernetes-26.1.0-2.el9.noar 410 kB/s | 1.0 MB 00:02 2024-03-14T21:11:46.269 INFO:teuthology.orchestra.run.smithi163.stdout:(123/156): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-03-14T21:11:46.319 INFO:teuthology.orchestra.run.smithi163.stdout:(124/156): python3-rsa-4.9-2.el9.noarch.rpm 284 kB/s | 59 kB 00:00 2024-03-14T21:11:46.412 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-14T21:11:46.494 INFO:teuthology.orchestra.run.smithi163.stdout:(125/156): python3-typing-extensions-4.4.0-2.el 227 kB/s | 51 kB 00:00 2024-03-14T21:11:46.728 INFO:teuthology.orchestra.run.smithi163.stdout:(126/156): python3-websocket-client-1.2.3-2.el9 385 kB/s | 90 kB 00:00 2024-03-14T21:11:46.878 INFO:teuthology.orchestra.run.smithi163.stdout:(127/156): python3-webob-1.8.7-6.el9.noarch.rpm 411 kB/s | 230 kB 00:00 2024-03-14T21:11:47.020 INFO:teuthology.orchestra.run.smithi163.stdout:(128/156): python3-zc-lockfile-2.0-10.el9.noarc 142 kB/s | 20 kB 00:00 2024-03-14T21:11:47.478 INFO:teuthology.orchestra.run.smithi163.stdout:(129/156): re2-20211101-3.el9.x86_64.rpm 419 kB/s | 192 kB 00:00 2024-03-14T21:11:47.795 INFO:teuthology.orchestra.run.smithi163.stdout:(130/156): python3-werkzeug-2.0.3-3.el9.1.noarc 400 kB/s | 427 kB 00:01 2024-03-14T21:11:48.237 INFO:teuthology.orchestra.run.smithi163.stdout:(131/156): librados2-18.2.2-852.g234d5323.el9.x 7.4 MB/s | 3.3 MB 00:00 2024-03-14T21:11:48.504 INFO:teuthology.orchestra.run.smithi163.stdout:(132/156): librbd1-18.2.2-852.g234d5323.el9.x86 11 MB/s | 3.0 MB 00:00 2024-03-14T21:11:48.546 INFO:teuthology.orchestra.run.smithi163.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-14T21:11:48.991 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T21:11:48.992 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-14T21:11:49.004 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-14T21:11:49.005 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-14T21:11:49.005 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-14T21:11:49.006 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-14T21:11:49.008 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-14T21:11:49.011 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-14T21:11:49.012 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-14T21:11:49.094 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-03-14T21:11:49.109 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================== 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================== 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:18.2.2-852.g234d5323.el9 ceph 6.5 k 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:18.2.2-852.g234d5323.el9 ceph 5.2 M 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:18.2.2-852.g234d5323.el9 ceph 843 k 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-852.g234d5323.el9 ceph 142 k 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:18.2.2-852.g234d5323.el9 ceph 1.4 M 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 138 k 2024-03-14T21:11:49.110 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 1.7 M 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 7.4 M 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 49 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:18.2.2-852.g234d5323.el9 ceph 7.7 M 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:18.2.2-852.g234d5323.el9 ceph 40 M 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 264 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 224 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:18.2.2-852.g234d5323.el9 ceph 222 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 18 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:18.2.2-852.g234d5323.el9 ceph 31 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 659 k 2024-03-14T21:11:49.111 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:18.2.2-852.g234d5323.el9 ceph 127 k 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:18.2.2-852.g234d5323.el9 ceph 161 k 2024-03-14T21:11:49.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:18.2.2-852.g234d5323.el9 ceph 321 k 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:18.2.2-852.g234d5323.el9 ceph 298 k 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:18.2.2-852.g234d5323.el9 ceph 99 k 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:18.2.2-852.g234d5323.el9 ceph 87 k 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.0 M 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:18.2.2-852.g234d5323.el9 ceph 172 k 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-14T21:11:49.113 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-14T21:11:49.114 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.3 M 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:18.2.2-852.g234d5323.el9 ceph 3.0 M 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-03-14T21:11:49.115 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:18.2.2-852.g234d5323.el9 ceph 18 M 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 23 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:18.2.2-852.g234d5323.el9 ceph 2.1 M 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 245 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:18.2.2-852.g234d5323.el9 ceph 4.5 M 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:18.2.2-852.g234d5323.el9 ceph 18 M 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:18.2.2-852.g234d5323.el9 ceph-noarch 15 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:18.2.2-852.g234d5323.el9 ceph 25 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-14T21:11:49.116 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:18.2.2-852.g234d5323.el9 ceph 166 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:18.2.2-852.g234d5323.el9 ceph 474 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:18.2.2-852.g234d5323.el9 ceph 4.5 M 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-14T21:11:49.117 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:18.2.2-852.g234d5323.el9 ceph 45 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:18.2.2-852.g234d5323.el9 ceph 127 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-14T21:11:49.118 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-14T21:11:49.119 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-14T21:11:49.120 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-14T21:11:49.121 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================== 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout:Install 132 Packages 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 24 Packages 2024-03-14T21:11:49.122 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:11:49.123 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 205 M 2024-03-14T21:11:49.123 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-03-14T21:11:49.146 INFO:teuthology.orchestra.run.smithi163.stdout:(134/156): NetworkManager-1.46.0-1.el9.x86_64.r 3.8 MB/s | 2.3 MB 00:00 2024-03-14T21:11:49.538 INFO:teuthology.orchestra.run.smithi049.stdout:(1/156): ceph-18.2.2-852.g234d5323.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2024-03-14T21:11:49.688 INFO:teuthology.orchestra.run.smithi163.stdout:(135/156): libarrow-9.0.0-10.el9.x86_64.rpm 421 kB/s | 4.4 MB 00:10 2024-03-14T21:11:49.747 INFO:teuthology.orchestra.run.smithi163.stdout:(136/156): NetworkManager-libnm-1.46.0-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-03-14T21:11:49.797 INFO:teuthology.orchestra.run.smithi163.stdout:(137/156): NetworkManager-tui-1.46.0-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-03-14T21:11:49.822 INFO:teuthology.orchestra.run.smithi163.stdout:(138/156): NetworkManager-team-1.46.0-1.el9.x86 319 kB/s | 42 kB 00:00 2024-03-14T21:11:49.848 INFO:teuthology.orchestra.run.smithi163.stdout:(139/156): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-14T21:11:49.881 INFO:teuthology.orchestra.run.smithi163.stdout:(140/156): elfutils-debuginfod-client-0.190-2.e 632 kB/s | 36 kB 00:00 2024-03-14T21:11:49.915 INFO:teuthology.orchestra.run.smithi163.stdout:(141/156): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-14T21:11:49.957 INFO:teuthology.orchestra.run.smithi163.stdout:(142/156): libacl-2.3.1-4.el9.x86_64.rpm 564 kB/s | 23 kB 00:00 2024-03-14T21:11:49.963 INFO:teuthology.orchestra.run.smithi049.stdout:(2/156): ceph-base-18.2.2-852.g234d5323.el9.x86 7.9 MB/s | 5.2 MB 00:00 2024-03-14T21:11:49.990 INFO:teuthology.orchestra.run.smithi163.stdout:(143/156): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-14T21:11:50.030 INFO:teuthology.orchestra.run.smithi049.stdout:(3/156): ceph-immutable-object-cache-18.2.2-852 2.1 MB/s | 142 kB 00:00 2024-03-14T21:11:50.041 INFO:teuthology.orchestra.run.smithi163.stdout:(144/156): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-03-14T21:11:50.074 INFO:teuthology.orchestra.run.smithi163.stdout:(145/156): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-14T21:11:50.100 INFO:teuthology.orchestra.run.smithi163.stdout:(146/156): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-14T21:11:50.125 INFO:teuthology.orchestra.run.smithi163.stdout:(147/156): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-03-14T21:11:50.150 INFO:teuthology.orchestra.run.smithi163.stdout:(148/156): libuuid-2.37.4-18.el9.x86_64.rpm 580 kB/s | 29 kB 00:00 2024-03-14T21:11:50.263 INFO:teuthology.orchestra.run.smithi049.stdout:(4/156): ceph-mds-18.2.2-852.g234d5323.el9.x86_ 9.1 MB/s | 2.1 MB 00:00 2024-03-14T21:11:50.375 INFO:teuthology.orchestra.run.smithi163.stdout:(149/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.6 MB/s | 2.2 MB 00:00 2024-03-14T21:11:50.405 INFO:teuthology.orchestra.run.smithi049.stdout:(5/156): ceph-fuse-18.2.2-852.g234d5323.el9.x86 972 kB/s | 843 kB 00:00 2024-03-14T21:11:50.434 INFO:teuthology.orchestra.run.smithi163.stdout:(150/156): openssl-3.0.7-27.el9.x86_64.rpm 3.9 MB/s | 1.2 MB 00:00 2024-03-14T21:11:50.439 INFO:teuthology.orchestra.run.smithi049.stdout:(6/156): ceph-mgr-18.2.2-852.g234d5323.el9.x86_ 8.3 MB/s | 1.4 MB 00:00 2024-03-14T21:11:50.493 INFO:teuthology.orchestra.run.smithi163.stdout:(151/156): thrift-0.15.0-2.el9.x86_64.rpm 537 kB/s | 1.6 MB 00:03 2024-03-14T21:11:50.593 INFO:teuthology.orchestra.run.smithi163.stdout:(152/156): util-linux-core-2.37.4-18.el9.x86_64 2.9 MB/s | 465 kB 00:00 2024-03-14T21:11:50.618 INFO:teuthology.orchestra.run.smithi163.stdout:(153/156): util-linux-user-2.37.4-18.el9.x86_64 250 kB/s | 31 kB 00:00 2024-03-14T21:11:50.718 INFO:teuthology.orchestra.run.smithi163.stdout:(154/156): util-linux-2.37.4-18.el9.x86_64.rpm 6.7 MB/s | 2.3 MB 00:00 2024-03-14T21:11:50.898 INFO:teuthology.orchestra.run.smithi049.stdout:(7/156): ceph-common-18.2.2-852.g234d5323.el9.x 12 MB/s | 18 MB 00:01 2024-03-14T21:11:50.927 INFO:teuthology.orchestra.run.smithi163.stdout:(155/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.2 MB/s | 1.1 MB 00:00 2024-03-14T21:11:51.069 INFO:teuthology.orchestra.run.smithi163.stdout:(156/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 165 kB/s | 74 kB 00:00 2024-03-14T21:11:51.075 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-03-14T21:11:51.075 INFO:teuthology.orchestra.run.smithi163.stdout:Total 8.6 MB/s | 205 MB 00:23 2024-03-14T21:11:51.856 INFO:teuthology.orchestra.run.smithi049.stdout:(8/156): ceph-osd-18.2.2-852.g234d5323.el9.x86_ 12 MB/s | 18 MB 00:01 2024-03-14T21:11:51.956 INFO:teuthology.orchestra.run.smithi049.stdout:(9/156): ceph-radosgw-18.2.2-852.g234d5323.el9. 7.3 MB/s | 7.7 MB 00:01 2024-03-14T21:11:51.981 INFO:teuthology.orchestra.run.smithi049.stdout:(10/156): ceph-selinux-18.2.2-852.g234d5323.el9 201 kB/s | 25 kB 00:00 2024-03-14T21:11:52.048 INFO:teuthology.orchestra.run.smithi049.stdout:(11/156): ceph-mon-18.2.2-852.g234d5323.el9.x86 2.7 MB/s | 4.5 MB 00:01 2024-03-14T21:11:52.073 INFO:teuthology.orchestra.run.smithi049.stdout:(12/156): cephfs-mirror-18.2.2-852.g234d5323.el 2.4 MB/s | 222 kB 00:00 2024-03-14T21:11:52.115 INFO:teuthology.orchestra.run.smithi049.stdout:(13/156): libcephfs-devel-18.2.2-852.g234d5323. 466 kB/s | 31 kB 00:00 2024-03-14T21:11:52.149 INFO:teuthology.orchestra.run.smithi049.stdout:(14/156): libcephfs2-18.2.2-852.g234d5323.el9.x 8.5 MB/s | 659 kB 00:00 2024-03-14T21:11:52.184 INFO:teuthology.orchestra.run.smithi049.stdout:(15/156): libcephsqlite-18.2.2-852.g234d5323.el 2.4 MB/s | 166 kB 00:00 2024-03-14T21:11:52.208 INFO:teuthology.orchestra.run.smithi049.stdout:(16/156): librados-devel-18.2.2-852.g234d5323.e 2.1 MB/s | 127 kB 00:00 2024-03-14T21:11:52.304 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-03-14T21:11:52.325 INFO:teuthology.orchestra.run.smithi049.stdout:(17/156): libradosstriper1-18.2.2-852.g234d5323 3.3 MB/s | 474 kB 00:00 2024-03-14T21:11:52.370 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-14T21:11:52.417 INFO:teuthology.orchestra.run.smithi049.stdout:(18/156): python3-ceph-argparse-18.2.2-852.g234 490 kB/s | 45 kB 00:00 2024-03-14T21:11:52.492 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-14T21:11:52.509 INFO:teuthology.orchestra.run.smithi049.stdout:(19/156): python3-ceph-common-18.2.2-852.g234d5 1.3 MB/s | 127 kB 00:00 2024-03-14T21:11:52.526 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-14T21:11:52.557 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-14T21:11:52.567 INFO:teuthology.orchestra.run.smithi049.stdout:(20/156): librgw2-18.2.2-852.g234d5323.el9.x86_ 12 MB/s | 4.5 MB 00:00 2024-03-14T21:11:52.592 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-14T21:11:52.593 INFO:teuthology.orchestra.run.smithi049.stdout:(21/156): python3-cephfs-18.2.2-852.g234d5323.e 1.9 MB/s | 161 kB 00:00 2024-03-14T21:11:52.618 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/180 2024-03-14T21:11:52.626 INFO:teuthology.orchestra.run.smithi049.stdout:(22/156): python3-rados-18.2.2-852.g234d5323.el 5.4 MB/s | 321 kB 00:00 2024-03-14T21:11:52.657 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/180 2024-03-14T21:11:52.673 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-03-14T21:11:52.673 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-03-14T21:11:52.677 INFO:teuthology.orchestra.run.smithi049.stdout:(23/156): python3-rbd-18.2.2-852.g234d5323.el9. 3.5 MB/s | 298 kB 00:00 2024-03-14T21:11:52.702 INFO:teuthology.orchestra.run.smithi049.stdout:(24/156): python3-rgw-18.2.2-852.g234d5323.el9. 1.3 MB/s | 99 kB 00:00 2024-03-14T21:11:52.702 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/180 2024-03-14T21:11:52.736 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-14T21:11:52.752 INFO:teuthology.orchestra.run.smithi049.stdout:(25/156): rbd-fuse-18.2.2-852.g234d5323.el9.x86 1.1 MB/s | 87 kB 00:00 2024-03-14T21:11:52.779 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-14T21:11:52.845 INFO:teuthology.orchestra.run.smithi049.stdout:(26/156): rbd-nbd-18.2.2-852.g234d5323.el9.x86_ 1.8 MB/s | 172 kB 00:00 2024-03-14T21:11:52.969 INFO:teuthology.orchestra.run.smithi049.stdout:(27/156): rbd-mirror-18.2.2-852.g234d5323.el9.x 11 MB/s | 3.0 MB 00:00 2024-03-14T21:11:52.987 INFO:teuthology.orchestra.run.smithi049.stdout:(28/156): ceph-grafana-dashboards-18.2.2-852.g2 165 kB/s | 23 kB 00:00 2024-03-14T21:11:53.020 INFO:teuthology.orchestra.run.smithi049.stdout:(29/156): ceph-mgr-cephadm-18.2.2-852.g234d5323 2.7 MB/s | 138 kB 00:00 2024-03-14T21:11:53.088 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-14T21:11:53.145 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:11:53.259 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:11:53.288 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:11:53.379 INFO:teuthology.orchestra.run.smithi049.stdout:(30/156): ceph-mgr-dashboard-18.2.2-852.g234d53 4.4 MB/s | 1.7 MB 00:00 2024-03-14T21:11:53.379 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:11:53.409 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 12/180 2024-03-14T21:11:53.446 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-14T21:11:53.487 INFO:teuthology.orchestra.run.smithi049.stdout:(31/156): ceph-mgr-modules-core-18.2.2-852.g234 2.2 MB/s | 245 kB 00:00 2024-03-14T21:11:53.579 INFO:teuthology.orchestra.run.smithi049.stdout:(32/156): ceph-mgr-rook-18.2.2-852.g234d5323.el 538 kB/s | 49 kB 00:00 2024-03-14T21:11:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-14T21:11:53.663 INFO:teuthology.orchestra.run.smithi049.stdout:(33/156): ceph-mgr-diskprediction-local-18.2.2- 12 MB/s | 7.4 MB 00:00 2024-03-14T21:11:53.680 INFO:teuthology.orchestra.run.smithi049.stdout:(34/156): ceph-prometheus-alerts-18.2.2-852.g23 148 kB/s | 15 kB 00:00 2024-03-14T21:11:53.707 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-14T21:11:53.735 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:11:53.747 INFO:teuthology.orchestra.run.smithi049.stdout:(35/156): cephadm-18.2.2-852.g234d5323.el9.noar 3.3 MB/s | 224 kB 00:00 2024-03-14T21:11:53.772 INFO:teuthology.orchestra.run.smithi049.stdout:(36/156): ceph-volume-18.2.2-852.g234d5323.el9. 2.4 MB/s | 264 kB 00:00 2024-03-14T21:11:53.819 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:11:53.830 INFO:teuthology.orchestra.run.smithi049.stdout:(37/156): cephfs-top-18.2.2-852.g234d5323.el9.n 217 kB/s | 18 kB 00:00 2024-03-14T21:11:53.845 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 17/180 2024-03-14T21:11:53.892 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 17/180 2024-03-14T21:11:53.906 INFO:teuthology.orchestra.run.smithi049.stdout:(38/156): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-03-14T21:11:53.933 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-14T21:11:53.939 INFO:teuthology.orchestra.run.smithi049.stdout:(39/156): bison-3.7.4-5.el9.x86_64.rpm 8.4 MB/s | 920 kB 00:00 2024-03-14T21:11:53.965 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-14T21:11:53.965 INFO:teuthology.orchestra.run.smithi049.stdout:(40/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-14T21:11:53.990 INFO:teuthology.orchestra.run.smithi049.stdout:(41/156): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-14T21:11:54.015 INFO:teuthology.orchestra.run.smithi049.stdout:(42/156): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-14T21:11:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-14T21:11:54.099 INFO:teuthology.orchestra.run.smithi049.stdout:(43/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-14T21:11:54.124 INFO:teuthology.orchestra.run.smithi049.stdout:(44/156): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-14T21:11:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:(45/156): libaio-devel-0.3.111-13.el9.x86_64.rp 305 kB/s | 10 kB 00:00 2024-03-14T21:11:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-14T21:11:54.183 INFO:teuthology.orchestra.run.smithi049.stdout:(46/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-14T21:11:54.217 INFO:teuthology.orchestra.run.smithi049.stdout:(47/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 754 kB/s | 44 kB 00:00 2024-03-14T21:11:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-14T21:11:54.250 INFO:teuthology.orchestra.run.smithi049.stdout:(48/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.6 MB/s | 661 kB 00:00 2024-03-14T21:11:54.284 INFO:teuthology.orchestra.run.smithi049.stdout:(49/156): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-03-14T21:11:54.305 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:11:54.309 INFO:teuthology.orchestra.run.smithi049.stdout:(50/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 794 kB/s | 46 kB 00:00 2024-03-14T21:11:54.370 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:11:54.657 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:11:54.668 INFO:teuthology.orchestra.run.smithi049.stdout:(51/156): ceph-test-18.2.2-852.g234d5323.el9.x8 15 MB/s | 40 MB 00:02 2024-03-14T21:11:54.691 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-14T21:11:54.693 INFO:teuthology.orchestra.run.smithi049.stdout:(52/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 691 kB/s | 282 kB 00:00 2024-03-14T21:11:54.718 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-14T21:11:54.718 INFO:teuthology.orchestra.run.smithi049.stdout:(53/156): oniguruma-6.9.6-1.el9.5.x86_64.rpm 530 kB/s | 217 kB 00:00 2024-03-14T21:11:54.743 INFO:teuthology.orchestra.run.smithi049.stdout:(54/156): openblas-0.3.21-2.el9.x86_64.rpm 462 kB/s | 34 kB 00:00 2024-03-14T21:11:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-14T21:11:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-14T21:11:54.878 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-03-14T21:11:54.878 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-03-14T21:11:54.900 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-14T21:11:54.935 INFO:teuthology.orchestra.run.smithi049.stdout:(55/156): postgresql-13.11-1.el9.x86_64.rpm 7.0 MB/s | 1.5 MB 00:00 2024-03-14T21:11:54.938 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-14T21:11:55.031 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-14T21:11:55.078 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-14T21:11:55.108 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-14T21:11:55.128 INFO:teuthology.orchestra.run.smithi049.stdout:(56/156): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-03-14T21:11:55.161 INFO:teuthology.orchestra.run.smithi049.stdout:(57/156): postgresql-contrib-13.11-1.el9.x86_64 1.9 MB/s | 811 kB 00:00 2024-03-14T21:11:55.186 INFO:teuthology.orchestra.run.smithi049.stdout:(58/156): postgresql-private-libs-13.11-1.el9.x 540 kB/s | 134 kB 00:00 2024-03-14T21:11:55.220 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-14T21:11:55.257 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:18.2.2-852.g234d5323.el9.x 33/180 2024-03-14T21:11:55.305 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:18.2.2-852.g234d5323.el9 34/180 2024-03-14T21:11:55.393 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 35/180 2024-03-14T21:11:55.520 INFO:teuthology.orchestra.run.smithi049.stdout:(59/156): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-03-14T21:11:55.536 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-14T21:11:55.545 INFO:teuthology.orchestra.run.smithi049.stdout:(60/156): python3-jsonpatch-1.21-16.el9.noarch. 68 kB/s | 24 kB 00:00 2024-03-14T21:11:55.591 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-14T21:11:55.770 INFO:teuthology.orchestra.run.smithi049.stdout:(61/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.9 MB/s | 5.8 MB 00:00 2024-03-14T21:11:55.798 INFO:teuthology.orchestra.run.smithi049.stdout:(62/156): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 18 kB 00:00 2024-03-14T21:11:55.823 INFO:teuthology.orchestra.run.smithi049.stdout:(63/156): python3-mako-1.1.4-6.el9.noarch.rpm 524 kB/s | 144 kB 00:00 2024-03-14T21:11:55.848 INFO:teuthology.orchestra.run.smithi049.stdout:(64/156): python3-markupsafe-1.1.1-12.el9.x86_6 406 kB/s | 32 kB 00:00 2024-03-14T21:11:55.874 INFO:teuthology.orchestra.run.smithi049.stdout:(65/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 217 kB 00:00 2024-03-14T21:11:56.066 INFO:teuthology.orchestra.run.smithi049.stdout:(66/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-14T21:11:56.091 INFO:teuthology.orchestra.run.smithi049.stdout:(67/156): python3-oauthlib-3.1.1-5.el9.noarch.r 720 kB/s | 174 kB 00:00 2024-03-14T21:11:56.116 INFO:teuthology.orchestra.run.smithi049.stdout:(68/156): python3-packaging-20.9-5.el9.noarch.r 285 kB/s | 69 kB 00:00 2024-03-14T21:11:56.142 INFO:teuthology.orchestra.run.smithi049.stdout:(69/156): python3-prettytable-0.7.2-27.el9.noar 550 kB/s | 41 kB 00:00 2024-03-14T21:11:56.151 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-14T21:11:56.225 INFO:teuthology.orchestra.run.smithi049.stdout:(70/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-03-14T21:11:56.228 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-14T21:11:56.250 INFO:teuthology.orchestra.run.smithi049.stdout:(71/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 211 kB 00:00 2024-03-14T21:11:56.276 INFO:teuthology.orchestra.run.smithi049.stdout:(72/156): python3-pytz-2021.1-5.el9.noarch.rpm 354 kB/s | 47 kB 00:00 2024-03-14T21:11:56.301 INFO:teuthology.orchestra.run.smithi049.stdout:(73/156): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2024-03-14T21:11:56.326 INFO:teuthology.orchestra.run.smithi049.stdout:(74/156): python3-toml-0.10.2-6.el9.noarch.rpm 774 kB/s | 39 kB 00:00 2024-03-14T21:11:56.333 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-14T21:11:56.380 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-14T21:11:56.394 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-14T21:11:56.435 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-14T21:11:56.473 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 43/180 2024-03-14T21:11:56.511 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-14T21:11:56.643 INFO:teuthology.orchestra.run.smithi049.stdout:(75/156): uuid-1.6.2-55.el9.x86_64.rpm 179 kB/s | 56 kB 00:00 2024-03-14T21:11:56.668 INFO:teuthology.orchestra.run.smithi049.stdout:(76/156): socat-1.7.4.1-5.el9.x86_64.rpm 817 kB/s | 300 kB 00:00 2024-03-14T21:11:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-14T21:11:56.942 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-14T21:11:56.994 INFO:teuthology.orchestra.run.smithi049.stdout:(77/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-03-14T21:11:57.006 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-14T21:11:57.019 INFO:teuthology.orchestra.run.smithi049.stdout:(78/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 167 kB/s | 63 kB 00:00 2024-03-14T21:11:57.044 INFO:teuthology.orchestra.run.smithi049.stdout:(79/156): zlib-devel-1.2.11-40.el9.x86_64.rpm 118 kB/s | 44 kB 00:00 2024-03-14T21:11:57.050 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-14T21:11:57.069 INFO:teuthology.orchestra.run.smithi049.stdout:(80/156): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-14T21:11:57.103 INFO:teuthology.orchestra.run.smithi049.stdout:(81/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-14T21:11:57.145 INFO:teuthology.orchestra.run.smithi049.stdout:(82/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-14T21:11:57.170 INFO:teuthology.orchestra.run.smithi049.stdout:(83/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-14T21:11:57.196 INFO:teuthology.orchestra.run.smithi049.stdout:(84/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-14T21:11:57.201 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-14T21:11:57.221 INFO:teuthology.orchestra.run.smithi049.stdout:(85/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-14T21:11:57.267 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-14T21:11:57.279 INFO:teuthology.orchestra.run.smithi049.stdout:(86/156): jq-1.6-16.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-03-14T21:11:57.352 INFO:teuthology.orchestra.run.smithi049.stdout:(87/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-03-14T21:11:57.380 INFO:teuthology.orchestra.run.smithi049.stdout:(88/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-03-14T21:11:57.382 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-14T21:11:57.422 INFO:teuthology.orchestra.run.smithi049.stdout:(89/156): python3-requests-2.25.1-8.el9.noarch. 877 kB/s | 125 kB 00:00 2024-03-14T21:11:57.430 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-14T21:11:57.487 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-14T21:11:57.560 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-14T21:11:57.597 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-14T21:11:57.633 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-14T21:11:57.682 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-14T21:11:57.730 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-14T21:11:57.789 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-14T21:11:57.930 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-14T21:11:57.989 INFO:teuthology.orchestra.run.smithi049.stdout:(90/156): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-03-14T21:11:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-14T21:11:58.047 INFO:teuthology.orchestra.run.smithi049.stdout:(91/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-03-14T21:11:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-14T21:11:58.182 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-14T21:11:58.223 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 64/180 2024-03-14T21:11:58.263 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-14T21:11:58.289 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-14T21:11:58.316 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-14T21:11:58.413 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-14T21:11:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-14T21:11:58.518 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-14T21:11:58.581 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-14T21:11:58.782 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-14T21:11:58.830 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-14T21:11:58.964 INFO:teuthology.orchestra.run.smithi049.stdout:(92/156): python3-devel-3.9.17-2.el9.x86_64.rpm 252 kB/s | 245 kB 00:00 2024-03-14T21:11:58.977 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-14T21:11:59.021 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 75/180 2024-03-14T21:11:59.022 INFO:teuthology.orchestra.run.smithi049.stdout:(93/156): python3-jinja2-2.11.3-5.el9.noarch.rp 254 kB/s | 248 kB 00:00 2024-03-14T21:11:59.061 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-14T21:11:59.100 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-14T21:11:59.129 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-14T21:11:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-14T21:11:59.200 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-14T21:11:59.230 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-14T21:11:59.309 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-14T21:11:59.340 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-14T21:11:59.373 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-14T21:11:59.415 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-14T21:11:59.450 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-14T21:11:59.498 INFO:teuthology.orchestra.run.smithi049.stdout:(94/156): fmt-8.1.1-5.el9.x86_64.rpm 233 kB/s | 111 kB 00:00 2024-03-14T21:11:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-14T21:11:59.586 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-14T21:11:59.634 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-14T21:11:59.731 INFO:teuthology.orchestra.run.smithi049.stdout:(95/156): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.3 MB/s | 309 kB 00:00 2024-03-14T21:11:59.744 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-14T21:11:59.805 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-14T21:11:59.856 INFO:teuthology.orchestra.run.smithi049.stdout:(96/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 105 kB/s | 94 kB 00:00 2024-03-14T21:12:00.032 INFO:teuthology.orchestra.run.smithi049.stdout:(97/156): libarrow-doc-9.0.0-10.el9.noarch.rpm 148 kB/s | 26 kB 00:00 2024-03-14T21:12:00.141 INFO:teuthology.orchestra.run.smithi049.stdout:(98/156): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-14T21:12:00.157 INFO:teuthology.orchestra.run.smithi049.stdout:(99/156): liboath-2.6.7-2.el9.x86_64.rpm 392 kB/s | 49 kB 00:00 2024-03-14T21:12:00.216 INFO:teuthology.orchestra.run.smithi049.stdout:(100/156): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-03-14T21:12:00.250 INFO:teuthology.orchestra.run.smithi049.stdout:(101/156): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-03-14T21:12:00.308 INFO:teuthology.orchestra.run.smithi049.stdout:(102/156): python3-asyncssh-2.13.2-4.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-03-14T21:12:00.334 INFO:teuthology.orchestra.run.smithi049.stdout:(103/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-03-14T21:12:00.384 INFO:teuthology.orchestra.run.smithi049.stdout:(104/156): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-03-14T21:12:00.409 INFO:teuthology.orchestra.run.smithi049.stdout:(105/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-14T21:12:00.468 INFO:teuthology.orchestra.run.smithi049.stdout:(106/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-14T21:12:00.493 INFO:teuthology.orchestra.run.smithi049.stdout:(107/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-14T21:12:00.552 INFO:teuthology.orchestra.run.smithi049.stdout:(108/156): python3-google-auth-2.28.0-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-14T21:12:00.570 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-14T21:12:00.577 INFO:teuthology.orchestra.run.smithi049.stdout:(109/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-03-14T21:12:00.618 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-14T21:12:00.619 INFO:teuthology.orchestra.run.smithi049.stdout:(110/156): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-03-14T21:12:00.644 INFO:teuthology.orchestra.run.smithi049.stdout:(111/156): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-03-14T21:12:00.660 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-14T21:12:00.686 INFO:teuthology.orchestra.run.smithi049.stdout:(112/156): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-03-14T21:12:00.701 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-14T21:12:00.720 INFO:teuthology.orchestra.run.smithi049.stdout:(113/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-03-14T21:12:00.779 INFO:teuthology.orchestra.run.smithi049.stdout:(114/156): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-14T21:12:00.804 INFO:teuthology.orchestra.run.smithi049.stdout:(115/156): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-03-14T21:12:00.846 INFO:teuthology.orchestra.run.smithi049.stdout:(116/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-14T21:12:00.877 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-14T21:12:00.880 INFO:teuthology.orchestra.run.smithi049.stdout:(117/156): python3-natsort-7.1.1-5.el9.noarch.r 763 kB/s | 58 kB 00:00 2024-03-14T21:12:00.907 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-14T21:12:00.922 INFO:teuthology.orchestra.run.smithi049.stdout:(118/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-14T21:12:00.947 INFO:teuthology.orchestra.run.smithi049.stdout:(119/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-03-14T21:12:00.989 INFO:teuthology.orchestra.run.smithi049.stdout:(120/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-14T21:12:00.993 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-14T21:12:01.014 INFO:teuthology.orchestra.run.smithi049.stdout:(121/156): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-03-14T21:12:01.058 INFO:teuthology.orchestra.run.smithi049.stdout:(122/156): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-03-14T21:12:01.081 INFO:teuthology.orchestra.run.smithi049.stdout:(123/156): python3-rsa-4.9-2.el9.noarch.rpm 882 kB/s | 59 kB 00:00 2024-03-14T21:12:01.123 INFO:teuthology.orchestra.run.smithi049.stdout:(124/156): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-03-14T21:12:01.148 INFO:teuthology.orchestra.run.smithi049.stdout:(125/156): python3-typing-extensions-4.4.0-2.el 764 kB/s | 51 kB 00:00 2024-03-14T21:12:01.191 INFO:teuthology.orchestra.run.smithi049.stdout:(126/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-03-14T21:12:01.216 INFO:teuthology.orchestra.run.smithi049.stdout:(127/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-14T21:12:01.266 INFO:teuthology.orchestra.run.smithi049.stdout:(128/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-03-14T21:12:01.292 INFO:teuthology.orchestra.run.smithi049.stdout:(129/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-14T21:12:01.333 INFO:teuthology.orchestra.run.smithi049.stdout:(130/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-14T21:12:01.400 INFO:teuthology.orchestra.run.smithi049.stdout:(131/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-03-14T21:12:01.551 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-03-14T21:12:01.679 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-14T21:12:01.690 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-14T21:12:01.700 INFO:teuthology.orchestra.run.smithi049.stdout:(132/156): librbd1-18.2.2-852.g234d5323.el9.x86 10 MB/s | 3.0 MB 00:00 2024-03-14T21:12:01.732 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-14T21:12:01.742 INFO:teuthology.orchestra.run.smithi049.stdout:(133/156): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-14T21:12:01.770 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-14T21:12:01.800 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-14T21:12:01.881 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-14T21:12:01.909 INFO:teuthology.orchestra.run.smithi049.stdout:(134/156): librados2-18.2.2-852.g234d5323.el9.x 5.7 MB/s | 3.3 MB 00:00 2024-03-14T21:12:01.927 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/180 2024-03-14T21:12:01.976 INFO:teuthology.orchestra.run.smithi049.stdout:(135/156): NetworkManager-1.46.0-1.el9.x86_64.r 9.8 MB/s | 2.3 MB 00:00 2024-03-14T21:12:02.004 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/180 2024-03-14T21:12:02.043 INFO:teuthology.orchestra.run.smithi049.stdout:(136/156): NetworkManager-team-1.46.0-1.el9.x86 635 kB/s | 42 kB 00:00 2024-03-14T21:12:02.055 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/180 2024-03-14T21:12:02.084 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-14T21:12:02.127 INFO:teuthology.orchestra.run.smithi049.stdout:(137/156): NetworkManager-tui-1.46.0-1.el9.x86_ 2.9 MB/s | 248 kB 00:00 2024-03-14T21:12:02.169 INFO:teuthology.orchestra.run.smithi049.stdout:(138/156): NetworkManager-libnm-1.46.0-1.el9.x8 7.0 MB/s | 1.8 MB 00:00 2024-03-14T21:12:02.181 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-14T21:12:02.227 INFO:teuthology.orchestra.run.smithi049.stdout:(139/156): acl-2.3.1-4.el9.x86_64.rpm 712 kB/s | 71 kB 00:00 2024-03-14T21:12:02.244 INFO:teuthology.orchestra.run.smithi049.stdout:(140/156): elfutils-debuginfod-client-0.190-2.e 485 kB/s | 36 kB 00:00 2024-03-14T21:12:02.303 INFO:teuthology.orchestra.run.smithi049.stdout:(141/156): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-14T21:12:02.336 INFO:teuthology.orchestra.run.smithi049.stdout:(142/156): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-14T21:12:02.370 INFO:teuthology.orchestra.run.smithi049.stdout:(143/156): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 23 kB 00:00 2024-03-14T21:12:02.403 INFO:teuthology.orchestra.run.smithi049.stdout:(144/156): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-14T21:12:02.445 INFO:teuthology.orchestra.run.smithi049.stdout:(145/156): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-14T21:12:02.488 INFO:teuthology.orchestra.run.smithi049.stdout:(146/156): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-14T21:12:02.513 INFO:teuthology.orchestra.run.smithi049.stdout:(147/156): libsmartcols-2.37.4-18.el9.x86_64.rp 952 kB/s | 64 kB 00:00 2024-03-14T21:12:02.555 INFO:teuthology.orchestra.run.smithi049.stdout:(148/156): libuuid-2.37.4-18.el9.x86_64.rpm 430 kB/s | 29 kB 00:00 2024-03-14T21:12:02.586 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-14T21:12:02.625 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:12:02.655 INFO:teuthology.orchestra.run.smithi049.stdout:(149/156): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-03-14T21:12:02.747 INFO:teuthology.orchestra.run.smithi049.stdout:(150/156): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-14T21:12:02.775 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:12:02.809 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:12:02.839 INFO:teuthology.orchestra.run.smithi049.stdout:(151/156): util-linux-core-2.37.4-18.el9.x86_64 5.0 MB/s | 465 kB 00:00 2024-03-14T21:12:02.889 INFO:teuthology.orchestra.run.smithi049.stdout:(152/156): util-linux-user-2.37.4-18.el9.x86_64 624 kB/s | 31 kB 00:00 2024-03-14T21:12:02.930 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:12:02.965 INFO:teuthology.orchestra.run.smithi049.stdout:(153/156): util-linux-2.37.4-18.el9.x86_64.rpm 7.4 MB/s | 2.3 MB 00:00 2024-03-14T21:12:02.976 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 12/180 2024-03-14T21:12:03.045 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-14T21:12:03.056 INFO:teuthology.orchestra.run.smithi049.stdout:(154/156): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.4 MB/s | 1.1 MB 00:00 2024-03-14T21:12:03.093 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-14T21:12:03.167 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-14T21:12:03.206 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-14T21:12:03.237 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-14T21:12:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-14T21:12:03.344 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-14T21:12:03.397 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-14T21:12:03.422 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:12:03.549 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:12:03.563 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-14T21:12:03.590 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 17/180 2024-03-14T21:12:03.595 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:18.2.2-852.g234d5323.el9.x86_64 107/180 2024-03-14T21:12:03.657 INFO:teuthology.orchestra.run.smithi049.stdout:(155/156): libacl-devel-2.3.1-4.el9.x86_64.rpm 107 kB/s | 74 kB 00:00 2024-03-14T21:12:03.662 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 17/180 2024-03-14T21:12:03.685 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:18.2.2-852.g234d5323.el9.x86_64 107/180 2024-03-14T21:12:03.716 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 108/180 2024-03-14T21:12:03.724 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-14T21:12:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-14T21:12:03.773 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-14T21:12:03.882 INFO:teuthology.orchestra.run.smithi049.stdout:(156/156): openssl-devel-3.0.7-27.el9.x86_64.rp 655 kB/s | 4.1 MB 00:06 2024-03-14T21:12:03.888 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-14T21:12:03.888 INFO:teuthology.orchestra.run.smithi049.stdout:Total 14 MB/s | 205 MB 00:14 2024-03-14T21:12:04.099 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-14T21:12:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-14T21:12:04.181 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:18.2.2-852.g234d5323.el9.x86_64 111/180 2024-03-14T21:12:04.231 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-14T21:12:04.252 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:18.2.2-852.g234d5323.el9.x86_64 111/180 2024-03-14T21:12:04.275 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 112/180 2024-03-14T21:12:04.302 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-03-14T21:12:04.310 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-14T21:12:04.524 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:12:04.575 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:12:04.591 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-03-14T21:12:04.591 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-03-14T21:12:05.004 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:12:05.047 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-14T21:12:05.086 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-14T21:12:05.172 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-14T21:12:05.303 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-14T21:12:05.376 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-14T21:12:05.433 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-14T21:12:05.536 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-14T21:12:05.579 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-14T21:12:05.612 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-14T21:12:05.720 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-14T21:12:05.758 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:18.2.2-852.g234d5323.el9.x 33/180 2024-03-14T21:12:05.808 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:18.2.2-852.g234d5323.el9 34/180 2024-03-14T21:12:05.913 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 35/180 2024-03-14T21:12:06.086 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-14T21:12:06.152 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-14T21:12:06.274 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-03-14T21:12:06.274 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-03-14T21:12:06.372 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:06.405 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:06.845 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-14T21:12:06.941 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-14T21:12:06.941 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:06.975 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 114/180 2024-03-14T21:12:07.040 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-14T21:12:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 114/180 2024-03-14T21:12:07.056 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-14T21:12:07.057 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-14T21:12:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:07.076 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-14T21:12:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 115/180 2024-03-14T21:12:07.116 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-14T21:12:07.173 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-14T21:12:07.219 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 43/180 2024-03-14T21:12:07.256 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-14T21:12:07.637 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-14T21:12:08.057 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-14T21:12:08.294 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-14T21:12:08.370 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-14T21:12:08.605 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-14T21:12:08.684 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-14T21:12:08.838 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-14T21:12:08.879 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-14T21:12:08.947 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-14T21:12:09.041 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-14T21:12:09.079 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-14T21:12:09.120 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-14T21:12:09.186 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-14T21:12:09.251 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-14T21:12:09.311 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-14T21:12:09.500 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-14T21:12:09.554 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-14T21:12:09.598 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-14T21:12:09.726 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-14T21:12:09.761 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 64/180 2024-03-14T21:12:09.807 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-14T21:12:09.843 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-14T21:12:09.882 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-14T21:12:10.032 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-14T21:12:10.092 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-14T21:12:10.162 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-14T21:12:10.241 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-14T21:12:10.491 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-14T21:12:10.563 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-14T21:12:10.742 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-14T21:12:10.797 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 75/180 2024-03-14T21:12:10.833 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-14T21:12:10.872 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-14T21:12:10.914 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-14T21:12:10.960 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-14T21:12:11.021 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-14T21:12:11.068 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-14T21:12:11.196 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-14T21:12:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-14T21:12:11.300 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-14T21:12:11.365 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-14T21:12:11.411 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-14T21:12:11.558 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-14T21:12:11.599 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-14T21:12:11.656 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-14T21:12:11.798 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-14T21:12:11.856 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-14T21:12:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-03-14T21:12:12.822 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-14T21:12:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/180 2024-03-14T21:12:12.872 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-14T21:12:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/180 2024-03-14T21:12:12.906 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-14T21:12:12.939 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-14T21:12:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-14T21:12:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/180 2024-03-14T21:12:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/180 2024-03-14T21:12:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/180 2024-03-14T21:12:13.079 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/180 2024-03-14T21:12:13.113 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/180 2024-03-14T21:12:13.156 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/180 2024-03-14T21:12:13.178 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-14T21:12:13.210 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-14T21:12:13.302 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-14T21:12:13.490 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/180 2024-03-14T21:12:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:12:13.639 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:12:13.665 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:12:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:12:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 12/180 2024-03-14T21:12:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/180 2024-03-14T21:12:14.095 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-14T21:12:14.142 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/180 2024-03-14T21:12:14.192 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/180 2024-03-14T21:12:14.228 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:12:14.247 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-14T21:12:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:12:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 17/180 2024-03-14T21:12:14.410 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 17/180 2024-03-14T21:12:14.458 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/180 2024-03-14T21:12:14.499 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 19/180 2024-03-14T21:12:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 20/180 2024-03-14T21:12:14.709 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-14T21:12:14.769 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 21/180 2024-03-14T21:12:14.838 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:12:14.896 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:12:15.174 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 22/180 2024-03-14T21:12:15.208 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/180 2024-03-14T21:12:15.235 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/180 2024-03-14T21:12:15.270 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/180 2024-03-14T21:12:15.347 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/180 2024-03-14T21:12:15.400 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/180 2024-03-14T21:12:15.437 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/180 2024-03-14T21:12:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/180 2024-03-14T21:12:15.544 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/180 2024-03-14T21:12:15.571 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/180 2024-03-14T21:12:15.643 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 32/180 2024-03-14T21:12:15.676 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:18.2.2-852.g234d5323.el9.x 33/180 2024-03-14T21:12:15.722 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:18.2.2-852.g234d5323.el9 34/180 2024-03-14T21:12:15.764 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-14T21:12:15.798 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 35/180 2024-03-14T21:12:15.874 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-14T21:12:15.914 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-14T21:12:15.946 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/180 2024-03-14T21:12:15.952 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-14T21:12:15.993 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-14T21:12:15.999 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/180 2024-03-14T21:12:16.354 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-14T21:12:16.390 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:18.2.2-852.g234d5323.el9.x86_64 107/180 2024-03-14T21:12:16.502 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:18.2.2-852.g234d5323.el9.x86_64 107/180 2024-03-14T21:12:16.538 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 108/180 2024-03-14T21:12:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/180 2024-03-14T21:12:16.592 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-14T21:12:16.648 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/180 2024-03-14T21:12:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/180 2024-03-14T21:12:16.771 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-14T21:12:16.783 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 41/180 2024-03-14T21:12:16.836 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/180 2024-03-14T21:12:16.874 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 43/180 2024-03-14T21:12:16.903 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/180 2024-03-14T21:12:17.037 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-14T21:12:17.084 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:18.2.2-852.g234d5323.el9.x86_64 111/180 2024-03-14T21:12:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 45/180 2024-03-14T21:12:17.168 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:18.2.2-852.g234d5323.el9.x86_64 111/180 2024-03-14T21:12:17.195 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 112/180 2024-03-14T21:12:17.462 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 46/180 2024-03-14T21:12:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 47/180 2024-03-14T21:12:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 48/180 2024-03-14T21:12:17.844 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/180 2024-03-14T21:12:17.891 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 50/180 2024-03-14T21:12:18.002 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 51/180 2024-03-14T21:12:18.037 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 52/180 2024-03-14T21:12:18.093 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/180 2024-03-14T21:12:18.165 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 115/180 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-03-14T21:12:18.166 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/180 2024-03-14T21:12:18.204 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/180 2024-03-14T21:12:18.240 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/180 2024-03-14T21:12:18.288 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/180 2024-03-14T21:12:18.340 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 116/180 2024-03-14T21:12:18.356 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 116/180 2024-03-14T21:12:18.357 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:18.357 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-14T21:12:18.357 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:18.375 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/180 2024-03-14T21:12:18.627 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/180 2024-03-14T21:12:18.767 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/180 2024-03-14T21:12:18.810 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/180 2024-03-14T21:12:18.852 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/180 2024-03-14T21:12:18.948 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/180 2024-03-14T21:12:18.984 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 64/180 2024-03-14T21:12:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/180 2024-03-14T21:12:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/180 2024-03-14T21:12:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 67/180 2024-03-14T21:12:19.184 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fmt-8.1.1-5.el9.x86_64 68/180 2024-03-14T21:12:19.237 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 69/180 2024-03-14T21:12:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 70/180 2024-03-14T21:12:19.351 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 71/180 2024-03-14T21:12:19.544 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 72/180 2024-03-14T21:12:19.591 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 73/180 2024-03-14T21:12:19.645 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:19.684 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:19.734 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 74/180 2024-03-14T21:12:19.773 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 75/180 2024-03-14T21:12:19.804 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/180 2024-03-14T21:12:19.835 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 77/180 2024-03-14T21:12:19.864 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 78/180 2024-03-14T21:12:19.898 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/180 2024-03-14T21:12:19.901 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 117/180 2024-03-14T21:12:19.927 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 117/180 2024-03-14T21:12:19.927 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:19.927 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-14T21:12:19.927 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T21:12:19.927 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T21:12:19.927 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:19.935 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/180 2024-03-14T21:12:19.964 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/180 2024-03-14T21:12:20.045 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/180 2024-03-14T21:12:20.075 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/180 2024-03-14T21:12:20.108 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/180 2024-03-14T21:12:20.144 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 118/180 2024-03-14T21:12:20.150 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/180 2024-03-14T21:12:20.166 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 118/180 2024-03-14T21:12:20.167 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:20.167 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-14T21:12:20.167 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T21:12:20.167 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T21:12:20.167 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:20.176 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/180 2024-03-14T21:12:20.262 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/180 2024-03-14T21:12:20.295 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/180 2024-03-14T21:12:20.297 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:20.338 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 114/180 2024-03-14T21:12:20.343 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/180 2024-03-14T21:12:20.475 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 90/180 2024-03-14T21:12:20.491 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 114/180 2024-03-14T21:12:20.492 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-14T21:12:20.492 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-14T21:12:20.492 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:20.531 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 115/180 2024-03-14T21:12:20.532 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 91/180 2024-03-14T21:12:20.613 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 119/180 2024-03-14T21:12:20.636 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 119/180 2024-03-14T21:12:20.636 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:20.636 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-14T21:12:20.636 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T21:12:20.636 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T21:12:20.636 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:20.740 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-14T21:12:21.087 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.2.2-852.g234d5323.el9.noarch 121/180 2024-03-14T21:12:21.114 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:18.2.2-852.g234d5323.el9.noarch 121/180 2024-03-14T21:12:21.143 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el 122/180 2024-03-14T21:12:21.208 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-852.g234d5323.e 123/180 2024-03-14T21:12:21.237 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 124/180 2024-03-14T21:12:21.299 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/180 2024-03-14T21:12:21.344 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 93/180 2024-03-14T21:12:21.377 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 94/180 2024-03-14T21:12:21.409 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 95/180 2024-03-14T21:12:21.464 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 124/180 2024-03-14T21:12:21.496 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 125/180 2024-03-14T21:12:21.594 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/180 2024-03-14T21:12:21.624 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/180 2024-03-14T21:12:21.704 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/180 2024-03-14T21:12:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/180 2024-03-14T21:12:22.465 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 125/180 2024-03-14T21:12:22.489 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 126/180 2024-03-14T21:12:22.491 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/180 2024-03-14T21:12:22.595 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 126/180 2024-03-14T21:12:22.754 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9 127/180 2024-03-14T21:12:22.791 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 128/180 2024-03-14T21:12:22.814 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 128/180 2024-03-14T21:12:22.814 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:22.814 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-14T21:12:22.814 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T21:12:22.814 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T21:12:22.814 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:22.875 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 129/180 2024-03-14T21:12:22.886 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 129/180 2024-03-14T21:12:22.932 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:18.2.2-852.g234d5323.el9.x86_64 130/180 2024-03-14T21:12:23.637 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-14T21:12:23.668 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 132/180 2024-03-14T21:12:23.691 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 132/180 2024-03-14T21:12:23.692 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:23.692 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-14T21:12:23.692 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T21:12:23.692 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T21:12:23.692 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:23.736 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-852.g234d53 133/180 2024-03-14T21:12:23.758 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-852.g234d53 133/180 2024-03-14T21:12:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:23.758 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-14T21:12:23.758 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:23.777 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/180 2024-03-14T21:12:23.803 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 134/180 2024-03-14T21:12:23.825 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 134/180 2024-03-14T21:12:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-14T21:12:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T21:12:23.826 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T21:12:23.826 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/180 2024-03-14T21:12:23.906 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 103/180 2024-03-14T21:12:23.936 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-16.el9.x86_64 104/180 2024-03-14T21:12:23.964 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 105/180 2024-03-14T21:12:24.120 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:24.143 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:24.143 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:24.143 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-14T21:12:24.144 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T21:12:24.144 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T21:12:24.144 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:24.278 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/180 2024-03-14T21:12:24.302 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:18.2.2-852.g234d5323.el9.x86_64 107/180 2024-03-14T21:12:24.393 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:18.2.2-852.g234d5323.el9.x86_64 107/180 2024-03-14T21:12:24.432 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 108/180 2024-03-14T21:12:24.468 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/180 2024-03-14T21:12:24.890 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/180 2024-03-14T21:12:24.921 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:18.2.2-852.g234d5323.el9.x86_64 111/180 2024-03-14T21:12:25.011 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:18.2.2-852.g234d5323.el9.x86_64 111/180 2024-03-14T21:12:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 112/180 2024-03-14T21:12:27.171 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:27.212 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:27.773 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 113/180 2024-03-14T21:12:27.807 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 114/180 2024-03-14T21:12:27.824 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 136/180 2024-03-14T21:12:27.865 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 137/180 2024-03-14T21:12:27.898 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 114/180 2024-03-14T21:12:27.898 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-14T21:12:27.898 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-14T21:12:27.898 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:27.904 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 138/180 2024-03-14T21:12:27.933 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 115/180 2024-03-14T21:12:28.096 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-14T21:12:28.211 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-14T21:12:28.255 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 141/180 2024-03-14T21:12:28.294 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-14T21:12:28.346 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-14T21:12:28.381 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-14T21:12:28.406 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_6 145/180 2024-03-14T21:12:28.446 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-14T21:12:28.476 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 147/180 2024-03-14T21:12:28.533 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 148/180 2024-03-14T21:12:28.613 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-14T21:12:28.650 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-14T21:12:29.178 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:29.240 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:30.117 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 152/180 2024-03-14T21:12:30.211 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-14T21:12:30.253 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-14T21:12:30.344 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-14T21:12:30.382 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-14T21:12:30.415 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-14T21:12:30.441 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-14T21:12:30.467 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-14T21:12:30.467 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-14T21:12:30.546 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-14T21:12:30.576 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-14T21:12:30.576 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-14T21:12:30.653 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-14T21:12:30.687 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-14T21:12:30.729 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-14T21:12:30.754 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-14T21:12:30.779 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-14T21:12:30.811 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-14T21:12:30.860 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-14T21:12:30.901 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-14T21:12:30.926 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-14T21:12:30.951 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-14T21:12:30.977 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-14T21:12:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-14T21:12:31.008 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:31.009 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:31.188 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:31.189 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-14T21:12:31.271 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-14T21:12:31.304 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-14T21:12:31.330 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-14T21:12:31.364 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-14T21:12:31.397 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-14T21:12:31.461 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-14T21:12:33.014 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 115/180 2024-03-14T21:12:33.014 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-03-14T21:12:33.014 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-03-14T21:12:33.014 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-03-14T21:12:33.014 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-03-14T21:12:33.014 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-03-14T21:12:33.015 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-03-14T21:12:33.015 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-03-14T21:12:33.015 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:33.256 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 116/180 2024-03-14T21:12:33.304 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 116/180 2024-03-14T21:12:33.305 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:33.305 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-14T21:12:33.305 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:35.026 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 117/180 2024-03-14T21:12:35.129 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 117/180 2024-03-14T21:12:35.129 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:35.129 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-14T21:12:35.129 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T21:12:35.129 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T21:12:35.129 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:35.351 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 118/180 2024-03-14T21:12:35.418 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 118/180 2024-03-14T21:12:35.419 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:35.419 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-14T21:12:35.419 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T21:12:35.419 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T21:12:35.419 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:18.2.2-852.g234d5323.el9.x86_64 1/180 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 2/180 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 3/180 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 4/180 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-852.g234d53 5/180 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 6/180 2024-03-14T21:12:35.924 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 7/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 8/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 9/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 12/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 13/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_6 14/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 15/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 17/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 18/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:18.2.2-852.g234d5323.el9.x86_64 19/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:18.2.2-852.g234d5323.el9 20/180 2024-03-14T21:12:35.925 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:18.2.2-852.g234d5323.el9.x 21/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 22/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 23/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 24/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 25/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 26/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 27/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 28/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-852.g234d5323.e 29/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 30/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 31/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 32/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9 33/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 34/180 2024-03-14T21:12:35.926 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el 35/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 36/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:18.2.2-852.g234d5323.el9.noarch 37/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 38/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-14T21:12:35.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-14T21:12:35.928 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-14T21:12:35.929 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-14T21:12:35.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 103/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-14T21:12:35.931 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 109/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-14T21:12:35.932 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-14T21:12:35.933 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:18.2.2-852.g234d5323.el9.x86_64 133/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 139/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 141/180 2024-03-14T21:12:35.934 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 143/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 145/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-14T21:12:35.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-14T21:12:35.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-14T21:12:35.937 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 119/180 2024-03-14T21:12:36.009 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 119/180 2024-03-14T21:12:36.009 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:36.009 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-14T21:12:36.009 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T21:12:36.009 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T21:12:36.009 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:36.129 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-14T21:12:36.735 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:18.2.2-852.g234d5323.el9.noarch 121/180 2024-03-14T21:12:36.764 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:18.2.2-852.g234d5323.el9.noarch 121/180 2024-03-14T21:12:36.811 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el 122/180 2024-03-14T21:12:36.925 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-852.g234d5323.e 123/180 2024-03-14T21:12:36.967 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 124/180 2024-03-14T21:12:37.286 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 124/180 2024-03-14T21:12:37.319 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 125/180 2024-03-14T21:12:38.448 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 125/180 2024-03-14T21:12:38.477 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 126/180 2024-03-14T21:12:38.618 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 126/180 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: acl-2.3.1-4.el9.x86_64 2024-03-14T21:12:38.627 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-14T21:12:38.628 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.629 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.630 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-16.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-14T21:12:38.631 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.632 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.11-1.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-14T21:12:38.633 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-14T21:12:38.634 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-14T21:12:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-14T21:12:38.636 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-14T21:12:38.637 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-14T21:12:38.638 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-14T21:12:38.639 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-14T21:12:38.639 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:12:38.639 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-14T21:12:38.784 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9 127/180 2024-03-14T21:12:38.830 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 128/180 2024-03-14T21:12:38.911 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 128/180 2024-03-14T21:12:38.911 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:38.911 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-14T21:12:38.912 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T21:12:38.912 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T21:12:38.912 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:38.972 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 129/180 2024-03-14T21:12:39.011 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 129/180 2024-03-14T21:12:39.105 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:18.2.2-852.g234d5323.el9.x86_64 130/180 2024-03-14T21:12:39.283 DEBUG:teuthology.parallel:result is None 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 115/180 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-03-14T21:12:39.765 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-03-14T21:12:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 116/180 2024-03-14T21:12:39.939 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-14T21:12:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 116/180 2024-03-14T21:12:39.943 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:39.943 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-14T21:12:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:39.983 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 132/180 2024-03-14T21:12:40.052 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 132/180 2024-03-14T21:12:40.052 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:40.052 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-14T21:12:40.052 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T21:12:40.052 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T21:12:40.052 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:40.100 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-852.g234d53 133/180 2024-03-14T21:12:40.166 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-852.g234d53 133/180 2024-03-14T21:12:40.167 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:40.167 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-14T21:12:40.167 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:40.234 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 134/180 2024-03-14T21:12:40.307 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 134/180 2024-03-14T21:12:40.307 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:40.307 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-14T21:12:40.307 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T21:12:40.307 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T21:12:40.308 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:40.665 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:40.734 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:40.734 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:40.734 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-14T21:12:40.734 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T21:12:40.734 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T21:12:40.734 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 117/180 2024-03-14T21:12:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 117/180 2024-03-14T21:12:41.581 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:41.581 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-14T21:12:41.581 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T21:12:41.581 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-14T21:12:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:41.797 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 118/180 2024-03-14T21:12:41.820 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 118/180 2024-03-14T21:12:41.820 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:41.820 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-14T21:12:41.820 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T21:12:41.821 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-14T21:12:41.821 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:42.290 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 119/180 2024-03-14T21:12:42.314 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 119/180 2024-03-14T21:12:42.314 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:42.314 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-14T21:12:42.314 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T21:12:42.314 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-14T21:12:42.314 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:42.408 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 120/180 2024-03-14T21:12:42.873 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:18.2.2-852.g234d5323.el9.noarch 121/180 2024-03-14T21:12:42.898 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:18.2.2-852.g234d5323.el9.noarch 121/180 2024-03-14T21:12:42.927 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el 122/180 2024-03-14T21:12:43.001 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-852.g234d5323.e 123/180 2024-03-14T21:12:43.030 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 124/180 2024-03-14T21:12:43.260 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 124/180 2024-03-14T21:12:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 125/180 2024-03-14T21:12:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 125/180 2024-03-14T21:12:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 126/180 2024-03-14T21:12:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 126/180 2024-03-14T21:12:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9 127/180 2024-03-14T21:12:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 128/180 2024-03-14T21:12:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 128/180 2024-03-14T21:12:44.640 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:44.641 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-14T21:12:44.641 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T21:12:44.641 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-14T21:12:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:44.690 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 136/180 2024-03-14T21:12:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 129/180 2024-03-14T21:12:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 129/180 2024-03-14T21:12:44.734 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 137/180 2024-03-14T21:12:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:18.2.2-852.g234d5323.el9.x86_64 130/180 2024-03-14T21:12:44.779 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 138/180 2024-03-14T21:12:45.035 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-14T21:12:45.154 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-14T21:12:45.207 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 141/180 2024-03-14T21:12:45.260 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-14T21:12:45.332 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-14T21:12:45.377 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-14T21:12:45.412 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_6 145/180 2024-03-14T21:12:45.457 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-14T21:12:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/180 2024-03-14T21:12:45.509 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 147/180 2024-03-14T21:12:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 132/180 2024-03-14T21:12:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 132/180 2024-03-14T21:12:45.550 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:45.550 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-14T21:12:45.550 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T21:12:45.550 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-14T21:12:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:45.594 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-852.g234d53 133/180 2024-03-14T21:12:45.596 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 148/180 2024-03-14T21:12:45.616 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-852.g234d53 133/180 2024-03-14T21:12:45.617 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:45.617 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-14T21:12:45.617 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:45.661 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 134/180 2024-03-14T21:12:45.685 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 134/180 2024-03-14T21:12:45.685 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:45.685 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-14T21:12:45.685 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T21:12:45.685 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-14T21:12:45.685 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:45.702 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-14T21:12:45.777 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-14T21:12:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-14T21:12:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-14T21:12:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T21:12:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-14T21:12:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:12:46.670 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:46.715 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:46.776 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:48.133 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 152/180 2024-03-14T21:12:48.231 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-14T21:12:48.279 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-14T21:12:48.407 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-14T21:12:48.442 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-14T21:12:48.475 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-14T21:12:48.509 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-14T21:12:48.550 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-14T21:12:48.551 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-14T21:12:48.693 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-14T21:12:49.106 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-14T21:12:49.106 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-14T21:12:49.274 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-14T21:12:49.321 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-14T21:12:49.367 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-14T21:12:49.400 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-14T21:12:49.433 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-14T21:12:49.503 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-14T21:12:49.581 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-14T21:12:49.622 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-14T21:12:49.654 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-14T21:12:49.688 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-14T21:12:49.722 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-14T21:12:49.755 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-14T21:12:49.791 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:49.792 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 136/180 2024-03-14T21:12:49.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 137/180 2024-03-14T21:12:49.970 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 138/180 2024-03-14T21:12:50.122 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:50.122 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-14T21:12:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/180 2024-03-14T21:12:50.262 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-14T21:12:50.272 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/180 2024-03-14T21:12:50.292 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-14T21:12:50.321 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 141/180 2024-03-14T21:12:50.325 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-14T21:12:50.360 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/180 2024-03-14T21:12:50.361 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-14T21:12:50.402 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-14T21:12:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 143/180 2024-03-14T21:12:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 144/180 2024-03-14T21:12:50.496 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_6 145/180 2024-03-14T21:12:50.521 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-14T21:12:50.537 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 146/180 2024-03-14T21:12:50.567 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 147/180 2024-03-14T21:12:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 148/180 2024-03-14T21:12:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/180 2024-03-14T21:12:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 150/180 2024-03-14T21:12:50.767 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-14T21:12:50.767 INFO:teuthology.orchestra.run.smithi163.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-14T21:12:50.767 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:51.383 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:51.422 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:51.442 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 151/180 2024-03-14T21:12:52.328 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 152/180 2024-03-14T21:12:52.409 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 153/180 2024-03-14T21:12:52.450 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/180 2024-03-14T21:12:52.533 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/180 2024-03-14T21:12:52.562 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/180 2024-03-14T21:12:52.587 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 157/180 2024-03-14T21:12:52.613 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 158/180 2024-03-14T21:12:52.664 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 159/180 2024-03-14T21:12:52.664 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-14T21:12:52.752 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/180 2024-03-14T21:12:52.781 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/180 2024-03-14T21:12:52.782 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-14T21:12:52.866 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/180 2024-03-14T21:12:52.893 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/180 2024-03-14T21:12:52.926 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 164/180 2024-03-14T21:12:52.951 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 165/180 2024-03-14T21:12:52.976 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 166/180 2024-03-14T21:12:53.009 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 167/180 2024-03-14T21:12:53.048 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 168/180 2024-03-14T21:12:53.081 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 169/180 2024-03-14T21:12:53.106 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 170/180 2024-03-14T21:12:53.131 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 171/180 2024-03-14T21:12:53.156 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 172/180 2024-03-14T21:12:53.181 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/180 2024-03-14T21:12:53.189 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:53.189 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 174/180 2024-03-14T21:12:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-14T21:12:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 175/180 2024-03-14T21:12:53.477 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 176/180 2024-03-14T21:12:53.503 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 177/180 2024-03-14T21:12:53.528 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 178/180 2024-03-14T21:12:53.554 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/180 2024-03-14T21:12:53.620 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-14T21:12:55.129 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:18.2.2-852.g234d5323.el9.x86_64 1/180 2024-03-14T21:12:55.129 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 2/180 2024-03-14T21:12:55.129 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 3/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 4/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-852.g234d53 5/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 6/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 7/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 8/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 9/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 12/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 13/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_6 14/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 15/180 2024-03-14T21:12:55.130 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 17/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 18/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:18.2.2-852.g234d5323.el9.x86_64 19/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:18.2.2-852.g234d5323.el9 20/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:18.2.2-852.g234d5323.el9.x 21/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 22/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 23/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 24/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 25/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 26/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 27/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 28/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-852.g234d5323.e 29/180 2024-03-14T21:12:55.131 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 30/180 2024-03-14T21:12:55.132 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 31/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 32/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9 33/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 34/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el 35/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 36/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:18.2.2-852.g234d5323.el9.noarch 37/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 38/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-14T21:12:55.145 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-14T21:12:55.146 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-14T21:12:55.147 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-14T21:12:55.148 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-14T21:12:55.149 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 103/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 109/180 2024-03-14T21:12:55.150 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-14T21:12:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:18.2.2-852.g234d5323.el9.x86_64 133/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-14T21:12:55.152 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 139/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 141/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 143/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 145/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-14T21:12:55.153 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-14T21:12:55.154 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-14T21:12:55.155 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-14T21:12:55.156 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-14T21:12:57.395 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: acl-2.3.1-4.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-14T21:12:57.396 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.397 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.2-8.el9.noarch 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.7.4-5.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.398 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.399 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.4-9.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-14T21:12:57.400 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.6-16.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.401 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-13.11-1.el9.x86_64 2024-03-14T21:12:57.402 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-14T21:12:57.403 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-14T21:12:57.404 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-14T21:12:57.405 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-14T21:12:57.406 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: re2-1:20211101-3.el9.x86_64 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-14T21:12:57.407 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-14T21:12:57.408 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-14T21:12:57.408 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-14T21:12:57.408 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-14T21:12:57.408 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-14T21:12:57.408 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:12:57.408 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-03-14T21:12:57.901 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 180/180 2024-03-14T21:12:57.901 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:18.2.2-852.g234d5323.el9.x86_64 1/180 2024-03-14T21:12:57.901 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 2/180 2024-03-14T21:12:57.901 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 3/180 2024-03-14T21:12:57.901 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 4/180 2024-03-14T21:12:57.901 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-852.g234d53 5/180 2024-03-14T21:12:57.901 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 6/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 7/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 8/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 9/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 10/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 11/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 12/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 13/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_6 14/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 15/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 16/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 17/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_ 18/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:18.2.2-852.g234d5323.el9.x86_64 19/180 2024-03-14T21:12:57.902 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:18.2.2-852.g234d5323.el9 20/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:18.2.2-852.g234d5323.el9.x 21/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 22/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 23/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 24/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 25/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 26/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 27/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 28/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-852.g234d5323.e 29/180 2024-03-14T21:12:57.903 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noar 30/180 2024-03-14T21:12:57.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.no 31/180 2024-03-14T21:12:57.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-852.g234d 32/180 2024-03-14T21:12:57.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9 33/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 34/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el 35/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 36/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:18.2.2-852.g234d5323.el9.noarch 37/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 38/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 42/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 43/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 44/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 45/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/180 2024-03-14T21:12:57.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 51/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 53/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 54/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 55/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.11-1.el9.x86_64 56/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 57/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 58/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 59/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/180 2024-03-14T21:12:57.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 72/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 74/180 2024-03-14T21:12:57.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 75/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 76/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 77/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 78/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 79/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 80/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 85/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-16.el9.x86_64 86/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 87/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 88/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 89/180 2024-03-14T21:12:57.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 90/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 91/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 92/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 93/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 94/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 95/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 96/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 102/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 103/180 2024-03-14T21:12:57.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 104/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 105/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 106/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 107/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 108/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 109/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 110/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 111/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 112/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 113/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 114/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/180 2024-03-14T21:12:57.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:18.2.2-852.g234d5323.el9.x86_64 133/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/180 2024-03-14T21:12:57.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:18.2.2-852.g234d5323.el9.x86_64 135/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 137/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 138/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 139/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 141/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 143/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 145/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 149/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/180 2024-03-14T21:12:57.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 151/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 153/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 157/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 158/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 159/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 160/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 161/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 162/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 163/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 164/180 2024-03-14T21:12:57.924 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 165/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 166/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 167/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 168/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 169/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 170/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 171/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 172/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 173/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 174/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 175/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 176/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 177/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 178/180 2024-03-14T21:12:57.925 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 179/180 2024-03-14T21:12:58.058 DEBUG:teuthology.parallel:result is None 2024-03-14T21:13:00.503 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 180/180 2024-03-14T21:13:00.503 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:00.503 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: acl-2.3.1-4.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.504 INFO:teuthology.orchestra.run.smithi049.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-03-14T21:13:00.505 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.506 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.507 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:18.2.2-852.g234d5323.el9.noarch 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-16.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-14T21:13:00.508 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-14T21:13:00.509 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.11-1.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-14T21:13:00.510 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-14T21:13:00.511 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-14T21:13:00.512 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-14T21:13:00.513 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-14T21:13:00.514 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:18.2.2-852.g234d5323.el9.x86_64 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-14T21:13:00.515 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-14T21:13:00.516 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-14T21:13:00.516 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-14T21:13:00.516 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:00.516 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-03-14T21:13:01.170 DEBUG:teuthology.parallel:result is None 2024-03-14T21:13:01.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:13:01.388 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-14T21:13:01.415 INFO:teuthology.orchestra.run.smithi049.stdout:18.2.2-852.g234d5323.el9 2024-03-14T21:13:01.415 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-852.g234d5323.el9 2024-03-14T21:13:01.416 INFO:teuthology.task.install:The correct ceph version 18.2.2-852.g234d5323 is installed. 2024-03-14T21:13:01.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:13:01.634 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-14T21:13:01.660 INFO:teuthology.orchestra.run.smithi097.stdout:18.2.2-852.g234d5323.el9 2024-03-14T21:13:01.660 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-852.g234d5323.el9 2024-03-14T21:13:01.660 INFO:teuthology.task.install:The correct ceph version 18.2.2-852.g234d5323 is installed. 2024-03-14T21:13:01.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:13:01.808 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-14T21:13:01.863 INFO:teuthology.orchestra.run.smithi163.stdout:18.2.2-852.g234d5323.el9 2024-03-14T21:13:01.863 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-852.g234d5323.el9 2024-03-14T21:13:01.863 INFO:teuthology.task.install:The correct ceph version 18.2.2-852.g234d5323 is installed. 2024-03-14T21:13:01.865 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-14T21:13:01.865 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:01.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-14T21:13:01.906 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:01.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-14T21:13:01.941 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:01.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-14T21:13:02.015 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-14T21:13:02.016 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:02.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-03-14T21:13:02.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-14T21:13:02.127 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:02.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-03-14T21:13:02.158 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-14T21:13:02.229 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:02.229 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-03-14T21:13:02.310 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-14T21:13:02.425 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-14T21:13:02.425 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:02.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-14T21:13:02.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-14T21:13:02.535 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:02.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-14T21:13:02.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-14T21:13:02.631 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:02.631 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-14T21:13:02.705 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-14T21:13:02.814 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-14T21:13:02.815 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:02.815 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-03-14T21:13:02.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-14T21:13:02.924 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:02.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-03-14T21:13:02.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-14T21:13:03.016 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:03.016 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-03-14T21:13:03.085 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-14T21:13:03.191 INFO:teuthology.run_tasks:Running task ceph... 2024-03-14T21:13:03.279 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-14T21:13:03.279 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /var/log/ceph 2024-03-14T21:13:03.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-03-14T21:13:03.284 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2024-03-14T21:13:03.349 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-14T21:13:03.349 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-14T21:13:03.358 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-14T21:13:03.360 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-14T21:13:03.428 INFO:tasks.ceph:Creating extra log directories... 2024-03-14T21:13:03.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-14T21:13:03.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-14T21:13:03.434 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-14T21:13:03.518 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-14T21:13:03.518 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-14T21:13:03.518 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-14T21:13:03.519 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:03.519 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-14T21:13:03.535 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:03.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:03.696 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:03.696 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-14T21:13:03.712 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:03.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:03.855 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:03.855 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-14T21:13:03.900 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:04.146 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:04.248 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-14T21:13:04.248 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-14T21:13:04.248 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405', 'branch': 'wip-yuri4-testing-2024-03-13-0733-reef', 'description': 'fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-dbench}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7600405', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2024-03-13-0733-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '234d532354068e06d01621fd032c3b663cead394'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '234d532354068e06d01621fd032c3b663cead394'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2024-03-13-0733-reef', 'sha1': '234d532354068e06d01621fd032c3b663cead394'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0']], 'seed': 1956, 'sha1': '234d532354068e06d01621fd032c3b663cead394', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2024-03-13-0733-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_234d532354068e06d01621fd032c3b663cead394/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '234d532354068e06d01621fd032c3b663cead394', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi049.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIsFzqFfZuzGWdRz7NTpDc8OGmCraRPPVX0whB8j8nGlLW2CZFDOrfhSltO7o0I8PMZJ5UojN0FCwrTlpMIbjJA=', 'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC8FSt3cq8kt44xgUYd26EGx1zagHHuP61WQJ0TefnB9toTrZAsEFMYq1EvrHvwIBAw+dx4UMwj6Tg5nT+YNDVY=', 'smithi163.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvawradYzO1Z9o2QO+uZ+V6JJs7yGcUFYNr2bd+3nXey6xgvubX6cevWGSStxIMBct5lZ/ACj4rJ4eA1VvS/8Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '234d532354068e06d01621fd032c3b663cead394'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '234d532354068e06d01621fd032c3b663cead394', 'cluster': 'ceph'}}, {'kclient': None}, {'workunit': {'clients': {'client.0': ['fs/fscrypt.sh none dbench', 'fs/fscrypt.sh unlocked dbench']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-14_15:28:28', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-14T21:13:04.249 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-14T21:13:04.251 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-14T21:13:04.253 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-14T21:13:04.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/run/ceph 2024-03-14T21:13:04.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-03-14T21:13:04.310 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2024-03-14T21:13:04.398 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:04.399 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-03-14T21:13:04.414 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-14T21:13:04.415 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-14 21:12:53.976440062 +0000 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-14 21:09:20.343436830 +0000 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-14 21:09:20.343436830 +0000 2024-03-14T21:13:04.471 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-14 21:09:20.343436830 +0000 2024-03-14T21:13:04.472 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-14T21:13:04.541 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-14T21:13:04.541 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-14T21:13:04.541 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000102308 s, 5.0 MB/s 2024-03-14T21:13:04.542 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-14T21:13:04.599 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-14 21:12:53.976440062 +0000 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-14 21:09:20.692426639 +0000 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-14 21:09:20.692426639 +0000 2024-03-14T21:13:04.655 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-14 21:09:20.692426639 +0000 2024-03-14T21:13:04.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-14T21:13:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-14T21:13:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-14T21:13:04.725 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000128342 s, 4.0 MB/s 2024-03-14T21:13:04.726 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-14T21:13:04.783 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-03-14T21:13:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-14T21:13:04.838 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:04.838 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-14T21:13:04.838 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:04.838 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:04.838 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-14 21:12:53.977440034 +0000 2024-03-14T21:13:04.839 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-14 21:09:21.021417032 +0000 2024-03-14T21:13:04.839 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-14 21:09:21.021417032 +0000 2024-03-14T21:13:04.839 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-14 21:09:21.021417032 +0000 2024-03-14T21:13:04.839 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-14T21:13:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-14T21:13:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-14T21:13:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.7252e-05 s, 5.3 MB/s 2024-03-14T21:13:04.909 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-14T21:13:04.965 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-03-14T21:13:05.020 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-14T21:13:05.020 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:05.020 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-14T21:13:05.020 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:05.020 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:05.021 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-14 21:12:53.977440034 +0000 2024-03-14T21:13:05.021 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-14 21:09:21.377406638 +0000 2024-03-14T21:13:05.021 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-14 21:09:21.377406638 +0000 2024-03-14T21:13:05.021 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-14 21:09:21.377406638 +0000 2024-03-14T21:13:05.021 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-14T21:13:05.089 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-14T21:13:05.089 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-14T21:13:05.089 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.4296e-05 s, 5.4 MB/s 2024-03-14T21:13:05.090 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-14T21:13:05.147 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:05.147 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:05.147 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-03-14T21:13:05.166 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-14T21:13:05.166 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-14 21:12:31.793603590 +0000 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-14 21:09:38.738245339 +0000 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-14 21:09:38.738245339 +0000 2024-03-14T21:13:05.223 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-14 21:09:38.738245339 +0000 2024-03-14T21:13:05.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-14T21:13:05.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-14T21:13:05.291 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-14T21:13:05.291 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000307393 s, 1.7 MB/s 2024-03-14T21:13:05.292 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-14T21:13:05.350 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-03-14T21:13:05.407 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-14T21:13:05.407 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:05.407 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-03-14T21:13:05.407 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:05.407 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:05.407 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-14 21:12:31.794603573 +0000 2024-03-14T21:13:05.407 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-14 21:09:39.079239334 +0000 2024-03-14T21:13:05.408 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-14 21:09:39.079239334 +0000 2024-03-14T21:13:05.408 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-14 21:09:39.079239334 +0000 2024-03-14T21:13:05.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-14T21:13:05.474 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-14T21:13:05.475 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-14T21:13:05.475 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000197387 s, 2.6 MB/s 2024-03-14T21:13:05.476 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-14T21:13:05.533 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-03-14T21:13:05.589 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-14T21:13:05.589 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:05.589 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-14T21:13:05.589 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:05.590 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:05.590 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-14 21:12:31.794603573 +0000 2024-03-14T21:13:05.590 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-14 21:09:39.412233469 +0000 2024-03-14T21:13:05.590 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-14 21:09:39.412233469 +0000 2024-03-14T21:13:05.590 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-14 21:09:39.412233469 +0000 2024-03-14T21:13:05.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-14T21:13:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-14T21:13:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-14T21:13:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000204617 s, 2.5 MB/s 2024-03-14T21:13:05.658 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-14T21:13:05.715 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-03-14T21:13:05.770 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-14T21:13:05.770 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T21:13:05.771 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-14T21:13:05.771 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:13:05.771 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T21:13:05.771 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-14 21:12:31.795603556 +0000 2024-03-14T21:13:05.771 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-14 21:09:39.744227622 +0000 2024-03-14T21:13:05.771 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-14 21:09:39.744227622 +0000 2024-03-14T21:13:05.771 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-14 21:09:39.744227622 +0000 2024-03-14T21:13:05.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-14T21:13:05.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-14T21:13:05.836 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-14T21:13:05.836 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000220851 s, 2.3 MB/s 2024-03-14T21:13:05.837 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-14T21:13:05.893 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:05.893 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-14T21:13:05.893 INFO:tasks.ceph:Generating config... 2024-03-14T21:13:05.894 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-14T21:13:05.895 INFO:tasks.ceph:[client] debug client = 20 2024-03-14T21:13:05.895 INFO:tasks.ceph:[client] debug ms = 1 2024-03-14T21:13:05.895 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-14T21:13:05.895 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-14T21:13:05.895 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-14T21:13:05.895 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-14T21:13:05.896 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-14T21:13:05.896 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-14T21:13:05.896 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-14T21:13:05.896 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-14T21:13:05.896 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-14T21:13:05.896 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-14T21:13:05.896 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-14T21:13:05.896 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-14T21:13:05.896 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-14T21:13:05.896 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-14T21:13:05.896 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-14T21:13:05.896 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-14T21:13:05.897 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-14T21:13:05.897 INFO:tasks.ceph:Setting up mon.a... 2024-03-14T21:13:05.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-14T21:13:05.952 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.keyring 2024-03-14T21:13:05.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-14T21:13:06.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T21:13:06.035 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.49'), ('mon.b', '172.21.15.97'), ('mon.c', '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]')] 2024-03-14T21:13:06.036 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.49,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]', 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-14T21:13:06.037 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:06.037 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-14T21:13:06.091 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.49 --add b 172.21.15.97 --addv c '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:06.176 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: generated fsid 25282add-1360-4a83-a2da-1a7e2b367f69 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = pacific 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 0 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:fsid 25282add-1360-4a83-a2da-1a7e2b367f69 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:last_changed 2024-03-14T21:13:06.194097+0000 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-03-14T21:13:06.194097+0000 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:min_mon_release 16 (pacific) 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:election_strategy: 1 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:2: [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] mon.c 2024-03-14T21:13:06.177 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-14T21:13:06.178 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-14T21:13:06.193 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 25282add-1360-4a83-a2da-1a7e2b367f69... 2024-03-14T21:13:06.193 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-14T21:13:06.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-14T21:13:06.237 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-14T21:13:06.288 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-03-14T21:13:06.288 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-03-14T21:13:06.288 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-14T21:13:06.288 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-03-14T21:13:06.289 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-03-14T21:13:06.290 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.49,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]" 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn min per osd = 0 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 25282add-1360-4a83-a2da-1a7e2b367f69 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-03-14T21:13:06.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-03-14T21:13:06.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2024-03-14T21:13:06.293 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.294 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-03-14T21:13:06.295 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-03-14T21:13:06.296 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-03-14T21:13:06.297 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-03-14T21:13:06.298 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2024-03-14T21:13:06.299 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2024-03-14T21:13:06.300 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2024-03-14T21:13:06.301 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]" 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn min per osd = 0 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 25282add-1360-4a83-a2da-1a7e2b367f69 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2024-03-14T21:13:06.302 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2024-03-14T21:13:06.303 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2024-03-14T21:13:06.304 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2024-03-14T21:13:06.305 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2024-03-14T21:13:06.306 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2024-03-14T21:13:06.308 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2024-03-14T21:13:06.309 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2024-03-14T21:13:06.310 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2024-03-14T21:13:06.310 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-03-14T21:13:06.310 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-03-14T21:13:06.310 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-03-14T21:13:06.411 INFO:teuthology.orchestra.run.smithi163.stdout:[global] 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: chdir = "" 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: auth supported = cephx 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: filestore xattr use omap = true 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: mon clock drift allowed = 1.000 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: osd crush chooseleaf type = 0 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: auth debug = true 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.412 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on old message = true 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on bug = true 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn max object skew = 0 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: # disable pg_autoscaler by default for new pools 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default size = 2 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow primary affinity = true 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow pg remap = true 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on legacy crush tunables = false 2024-03-14T21:13:06.413 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on crush straw calc version zero = false 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on no sortbitwise = false 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on osd down out interval zero = false 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on too few osds = false 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon_allow_pool_size_one = true 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: osd default data pool replay window = 5 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon allow pool delete = true 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: mon cluster log file level = debug 2024-03-14T21:13:06.414 INFO:teuthology.orchestra.run.smithi163.stdout: debug asserts on shutdown = true 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: mon health detail to clog = false 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: mon host = "172.21.15.49,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]" 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn min per osd = 0 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: fsid = 25282add-1360-4a83-a2da-1a7e2b367f69 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout:[osd] 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: osd journal size = 100 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub load threshold = 5.0 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub max interval = 600 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: osd mclock profile = high_recovery_ops 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: osd recover clone overlap = true 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: osd recovery max chunk = 1048576 2024-03-14T21:13:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug shutdown = true 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug op order = true 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug verify stray on activate = true 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: osd open classes on start = true 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug pg log writeout = true 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: osd deep scrub update digest min age = 30 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: osd map max advance = 10 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: journal zero on create = true 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: filestore ondisk finisher threads = 3 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: filestore apply finisher threads = 3 2024-03-14T21:13:06.416 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: bdev debug aio = true 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug misdirected ops = true 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: bdev async discard = True 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: bdev enable discard = True 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore allocator = bitmap 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore block size = 96636764160 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore fsck on mount = True 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluefs = 1/20 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluestore = 1/20 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: debug osd = 20 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: debug rocksdb = 4/10 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd backfillfull_ratio = 0.85 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd full ratio = 0.9 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd nearfull ratio = 0.8 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: osd failsafe full ratio = 0.95 2024-03-14T21:13:06.417 INFO:teuthology.orchestra.run.smithi163.stdout: osd objectstore = bluestore 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr] 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: debug mgr = 20 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: mgr/telemetry/nag = false 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout:[mon] 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-14T21:13:06.418 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: debug paxos = 20 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon data avail warn = 5 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon mgr mkfs grace = 240 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd reporter subtree level = osd 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd prime pg temp = true 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: auth mon ticket ttl = 660 # 11m 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: auth service ticket ttl = 240 # 4m 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: # don't complain about insecure global_id in the test suite 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-14T21:13:06.419 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: # 1m isn't quite enough 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: mon_down_mkfs_grace = 2m 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_filestore_osds = false 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: mon op complaint time = 120 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout:[client] 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: rgw cache enabled = true 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable ops log = true 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable usage log = true 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: client mount timeout = 600 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-14T21:13:06.420 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.a] 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.b] 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.c] 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout:[mds] 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds = 20 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds balancer = 20 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug frag = True 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug scatterstat = True 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: mds op complaint time = 180 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: mds verify scatter = True 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-03-14T21:13:06.421 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-03-14T21:13:06.422 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-03-14T21:13:06.453 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-14T21:13:06.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-14T21:13:06.513 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-14T21:13:06.514 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:06.514 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-14T21:13:06.528 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:06.528 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-14T21:13:06.583 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T21:13:06.583 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:06.583 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-14T21:13:06.583 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T21:13:06.669 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:06.669 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:06.724 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T21:13:06.724 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:06.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-14T21:13:06.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T21:13:06.766 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:06.766 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:06.822 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2024-03-14T21:13:06.822 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:06.822 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-14T21:13:06.822 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T21:13:06.972 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:06.972 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:07.069 INFO:tasks.ceph:Setting up mon nodes... 2024-03-14T21:13:07.069 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-14T21:13:07.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-14T21:13:07.133 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-14T21:13:07.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-14T21:13:07.222 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-14T21:13:07.225 INFO:tasks.ceph:Setting up mds nodes... 2024-03-14T21:13:07.225 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-14T21:13:07.288 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-14T21:13:07.291 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-14T21:13:07.321 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-14T21:13:07.422 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-14T21:13:07.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-14T21:13:07.455 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-14T21:13:07.512 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-14T21:13:07.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-14T21:13:07.541 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-14T21:13:07.541 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-14T21:13:07.674 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-14T21:13:07.731 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-14T21:13:07.731 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-14T21:13:07.731 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-14T21:13:07.762 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:07.762 INFO:tasks.ceph:role: osd.0 2024-03-14T21:13:07.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T21:13:07.762 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:07.833 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:07.872 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-03-14T21:13:07.874 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-03-14T21:13:07.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-14T21:13:07.950 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-14T21:13:08.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-14T21:13:08.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:08.096 INFO:tasks.ceph:role: osd.1 2024-03-14T21:13:08.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T21:13:08.096 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-14T21:13:08.166 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:08.166 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:08.166 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:08.166 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:08.167 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:08.167 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:08.167 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:08.167 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:08.167 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:08.167 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:08.205 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-03-14T21:13:08.207 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-03-14T21:13:08.207 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-14T21:13:08.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-14T21:13:08.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-14T21:13:08.427 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:08.428 INFO:tasks.ceph:role: osd.2 2024-03-14T21:13:08.428 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T21:13:08.428 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:08.498 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:08.499 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:08.539 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-03-14T21:13:08.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-03-14T21:13:08.541 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-14T21:13:08.618 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-14T21:13:08.693 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-14T21:13:08.765 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:08.765 INFO:tasks.ceph:role: osd.3 2024-03-14T21:13:08.765 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T21:13:08.765 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-14T21:13:08.835 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:08.835 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:08.835 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:08.835 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:08.835 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:08.835 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:08.835 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:08.836 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:08.836 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:08.836 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:08.874 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-03-14T21:13:08.876 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-03-14T21:13:08.876 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-14T21:13:08.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-14T21:13:09.027 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:09.154+0000 7f20c3038500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-14T21:13:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:09.154+0000 7f20c3038500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-14T21:13:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:09.154+0000 7f20c3038500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T21:13:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:09.154+0000 7f20c3038500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T21:13:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:09.154+0000 7f20c3038500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T21:13:09.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:09.154+0000 7f20c3038500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-14T21:13:11.984 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-14T21:13:12.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:12.126+0000 7f62da60e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-14T21:13:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:12.126+0000 7f62da60e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-14T21:13:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:12.126+0000 7f62da60e500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T21:13:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:12.127+0000 7f62da60e500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T21:13:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:12.127+0000 7f62da60e500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T21:13:12.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:12.127+0000 7f62da60e500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-14T21:13:14.961 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-14T21:13:14.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:15.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:15.106+0000 7f0d4f60c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-14T21:13:15.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:15.106+0000 7f0d4f60c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-14T21:13:15.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:15.106+0000 7f0d4f60c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T21:13:15.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:15.106+0000 7f0d4f60c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T21:13:15.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:15.106+0000 7f0d4f60c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T21:13:15.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:15.107+0000 7f0d4f60c500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-14T21:13:17.956 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-14T21:13:17.992 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:18.102+0000 7fb056d6c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-14T21:13:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:18.103+0000 7fb056d6c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-14T21:13:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:18.103+0000 7fb056d6c500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T21:13:18.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:18.103+0000 7fb056d6c500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T21:13:18.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:18.103+0000 7fb056d6c500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T21:13:18.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:18.103+0000 7fb056d6c500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-14T21:13:20.957 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-14T21:13:20.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-14T21:13:21.024 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:21.024 INFO:tasks.ceph:role: osd.4 2024-03-14T21:13:21.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T21:13:21.025 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-14T21:13:21.101 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:21.101 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:21.101 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:21.101 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:21.102 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:21.102 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:21.102 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:21.102 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:21.102 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:21.102 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:21.615 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-14T21:13:21.618 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-14T21:13:21.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-14T21:13:21.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-14T21:13:21.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-14T21:13:21.872 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:21.873 INFO:tasks.ceph:role: osd.5 2024-03-14T21:13:21.873 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T21:13:21.873 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:21.948 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:22.397 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-14T21:13:22.399 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-14T21:13:22.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-14T21:13:22.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-14T21:13:22.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-14T21:13:22.603 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:22.603 INFO:tasks.ceph:role: osd.6 2024-03-14T21:13:22.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T21:13:22.603 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:22.678 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:22.679 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:22.679 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:23.184 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-14T21:13:23.186 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-14T21:13:23.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-14T21:13:23.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-14T21:13:23.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-14T21:13:23.450 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T21:13:23.450 INFO:tasks.ceph:role: osd.7 2024-03-14T21:13:23.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T21:13:23.451 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T21:13:23.526 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T21:13:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-14T21:13:24.051 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-14T21:13:24.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-14T21:13:24.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-14T21:13:24.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:24.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:24.310+0000 7fb7e1cfa500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-14T21:13:24.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:24.310+0000 7fb7e1cfa500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-14T21:13:24.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:24.310+0000 7fb7e1cfa500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T21:13:24.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:24.310+0000 7fb7e1cfa500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T21:13:24.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:24.310+0000 7fb7e1cfa500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T21:13:24.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:24.311+0000 7fb7e1cfa500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-14T21:13:27.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-14T21:13:27.175 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:27.278+0000 7f3e1e594500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-14T21:13:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:27.278+0000 7f3e1e594500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-14T21:13:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:27.279+0000 7f3e1e594500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T21:13:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:27.279+0000 7f3e1e594500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T21:13:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:27.279+0000 7f3e1e594500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T21:13:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:27.279+0000 7f3e1e594500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-14T21:13:30.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-14T21:13:30.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:30.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:30.250+0000 7f7cd2507500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-14T21:13:30.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:30.250+0000 7f7cd2507500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-14T21:13:30.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:30.250+0000 7f7cd2507500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T21:13:30.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:30.250+0000 7f7cd2507500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T21:13:30.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:30.250+0000 7f7cd2507500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T21:13:30.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:30.250+0000 7f7cd2507500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-14T21:13:33.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-14T21:13:33.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:33.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:33.216+0000 7f80d49f9500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-14T21:13:33.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:33.216+0000 7f80d49f9500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-14T21:13:33.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:33.216+0000 7f80d49f9500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T21:13:33.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:33.216+0000 7f80d49f9500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T21:13:33.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:33.216+0000 7f80d49f9500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T21:13:33.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:33.216+0000 7f80d49f9500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-14T21:13:34.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:34.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:34.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:13:36.043 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-14T21:13:36.072 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-14T21:13:36.072 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:36.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-14T21:13:36.108 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:36.108 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-14T21:13:36.182 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:36.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-14T21:13:36.255 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:36.255 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-14T21:13:36.328 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:36.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-14T21:13:36.400 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:36.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-14T21:13:36.473 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:36.473 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-14T21:13:36.544 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:36.544 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-14T21:13:36.576 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:36.576 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-14T21:13:36.647 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:36.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-14T21:13:36.717 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:36.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-14T21:13:36.787 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:36.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-14T21:13:36.856 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:36.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-14T21:13:36.924 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:13:36.925 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-14T21:13:36.981 INFO:tasks.ceph:Adding keys to all mons... 2024-03-14T21:13:36.981 DEBUG:teuthology.orchestra.run.smithi049:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-14T21:13:36.984 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBjaPNlYxsDCRAAqOzxL/Bn20N3mzI29oDhbg== 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBjaPNl4RZBEhAAmWTtRgxjjumRQAriEc0TFg== 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBjaPNl9+w8GhAA2lLCJy7dsvW71LKF81GFZQ== 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBlaPNl0sk6CRAAHGvZ7OxH2qQE4+d24vcSMw== 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBoaPNlS8ilBxAA+TmlkIsZTtnx/NMpXJXWyA== 2024-03-14T21:13:37.010 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBraPNlbAV0BhAAYjqlZ1rWa3GZQYA/WbWSIw== 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBuaPNlf9U3BhAAN4MGkp4DuKSSY9dqkdbITQ== 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBjaPNlaaxQDhAA+D4bOfz9VudrWfuvO4rIVQ== 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBjaPNlW1WQHxAAJnMyhmMom632ZC0JxNmCtA== 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQB0aPNl3KCMEhAAnzLxQL9tvbBwpCU0HBQvSQ== 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQB3aPNlL/KqEBAAiHbm+VGS88gpFRJS1RGX7Q== 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQB6aPNltET3DhAACU/oBinEykk2QkAb6MrlNQ== 2024-03-14T21:13:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2024-03-14T21:13:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQB9aPNl+1zvDBAAXaWjkgZ482+jdkV6EPWL+Q== 2024-03-14T21:13:37.012 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-03-14T21:13:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBjaPNl2FQ+KRAA9KZFge1Cqzl2smwLW2AArw== 2024-03-14T21:13:37.013 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.y] 2024-03-14T21:13:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBjaPNlYxsDCRAAqOzxL/Bn20N3mzI29oDhbg== 2024-03-14T21:13:37.013 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-03-14T21:13:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBjaPNl4RZBEhAAmWTtRgxjjumRQAriEc0TFg== 2024-03-14T21:13:37.013 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-03-14T21:13:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBjaPNl9+w8GhAA2lLCJy7dsvW71LKF81GFZQ== 2024-03-14T21:13:37.014 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.0] 2024-03-14T21:13:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBlaPNl0sk6CRAAHGvZ7OxH2qQE4+d24vcSMw== 2024-03-14T21:13:37.014 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.1] 2024-03-14T21:13:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBoaPNlS8ilBxAA+TmlkIsZTtnx/NMpXJXWyA== 2024-03-14T21:13:37.014 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.2] 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBraPNlbAV0BhAAYjqlZ1rWa3GZQYA/WbWSIw== 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.3] 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBuaPNlf9U3BhAAN4MGkp4DuKSSY9dqkdbITQ== 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.x] 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBjaPNlaaxQDhAA+D4bOfz9VudrWfuvO4rIVQ== 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBjaPNlW1WQHxAAJnMyhmMom632ZC0JxNmCtA== 2024-03-14T21:13:37.015 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.4] 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB0aPNl3KCMEhAAnzLxQL9tvbBwpCU0HBQvSQ== 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.5] 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB3aPNlL/KqEBAAiHbm+VGS88gpFRJS1RGX7Q== 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.6] 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB6aPNltET3DhAACU/oBinEykk2QkAb6MrlNQ== 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.7] 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB9aPNl+1zvDBAAXaWjkgZ482+jdkV6EPWL+Q== 2024-03-14T21:13:37.016 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-03-14T21:13:37.017 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBjaPNl2FQ+KRAA9KZFge1Cqzl2smwLW2AArw== 2024-03-14T21:13:37.019 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T21:13:37.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T21:13:37.117 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T21:13:37.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T21:13:37.176 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T21:13:37.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T21:13:37.234 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.290 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.456 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T21:13:37.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T21:13:37.512 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T21:13:37.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T21:13:37.571 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.573 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.682 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T21:13:37.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T21:13:37.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T21:13:37.853 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-14T21:13:37.853 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-14T21:13:37.884 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T21:13:37.985 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-14T21:13:38.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-14T21:13:38.042 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T21:13:38.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-14T21:13:38.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-14T21:13:38.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T21:13:38.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-14T21:13:38.356 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:38.358 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-14T21:13:38.413 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-14T21:13:38.413 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-14T21:13:38.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-14T21:13:38.416 INFO:tasks.ceph.mon.a:Started 2024-03-14T21:13:38.416 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-14T21:13:38.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-14T21:13:38.454 INFO:tasks.ceph.mon.b:Started 2024-03-14T21:13:38.454 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-14T21:13:38.454 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-14T21:13:38.456 INFO:tasks.ceph.mon.c:Started 2024-03-14T21:13:38.456 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-14T21:13:38.456 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-14T21:13:38.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-14T21:13:38.459 INFO:tasks.ceph.mgr.y:Started 2024-03-14T21:13:38.460 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-14T21:13:38.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-14T21:13:38.462 INFO:tasks.ceph.mgr.x:Started 2024-03-14T21:13:38.462 INFO:tasks.ceph:Setting crush tunables to default 2024-03-14T21:13:38.462 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd crush tunables default 2024-03-14T21:13:38.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.585+0000 7f62a813c640 1 Processor -- start 2024-03-14T21:13:38.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.585+0000 7f62a813c640 1 -- start start 2024-03-14T21:13:38.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a813c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:38.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a813c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a006d670 con 0x7f62a00716f0 2024-03-14T21:13:38.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a813c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a00b8c40 con 0x7f62a00b9930 2024-03-14T21:13:38.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a813c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a00b8f20 con 0x7f62a006cf10 2024-03-14T21:13:38.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a66d9640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f62a00716f0 0x7f62a01520b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:34066/0 (socket says 172.21.15.49:34066) 2024-03-14T21:13:38.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a66d9640 1 -- 172.21.15.49:0/129436798 learned_addr learned my addr 172.21.15.49:0/129436798 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:13:38.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a5ed8640 1 -- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 msgr2=0x7f62a0147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-14T21:13:38.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a5ed8640 1 -- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 msgr2=0x7f62a0147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-14T21:13:38.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-14T21:13:38.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.586+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-14T21:13:38.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.787+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.787+0000 7f62a5ed8640 1 -- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 msgr2=0x7f62a0147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-14T21:13:38.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.787+0000 7f62a5ed8640 1 -- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 msgr2=0x7f62a0147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-14T21:13:38.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.787+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-14T21:13:38.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.787+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-14T21:13:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.792+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136620208 0 0) 0x7f62a006d670 con 0x7f62a00716f0 2024-03-14T21:13:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.792+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f628c003520 con 0x7f62a00716f0 2024-03-14T21:13:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721014218 0 0) 0x7f62a00b8c40 con 0x7f62a00b9930 2024-03-14T21:13:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a006d670 con 0x7f62a00b9930 2024-03-14T21:13:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2065792737 0 0) 0x7f62a006d670 con 0x7f62a00b9930 2024-03-14T21:13:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 msgr2=0x7f62a0147c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T21:13:38.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 --2- 172.21.15.49:0/129436798 >> v2:172.21.15.97:3300/0 conn(0x7f62a006cf10 0x7f62a0147c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 >> v1:172.21.15.49:6789/0 conn(0x7f62a00716f0 legacy=0x7f62a01520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00b9200 con 0x7f62a00b9930 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f629c001670 con 0x7f62a00b9930 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 >> v1:172.21.15.97:6789/0 conn(0x7f62a00b9930 legacy=0x7f62a01415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628c003b00 0x7f628c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c0047c0 0x7f628c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f628c005180 0x7f628c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00b8f20 con 0x7f628c005180 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c00c0b0 con 0x7f628c003b00 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c00c270 con 0x7f628c0047c0 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a66d9640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f628c005180 0x7f628c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628c003b00 0x7f628c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a56d7640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c0047c0 0x7f628c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a66d9640 1 -- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c0047c0 msgr2=0x7f628c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a66d9640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c0047c0 0x7f628c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a66d9640 1 -- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628c003b00 msgr2=0x7f628c003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a66d9640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f628c003b00 0x7f628c003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.793+0000 7f62a66d9640 1 -- 172.21.15.49:0/129436798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a006d670 con 0x7f628c005180 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.794+0000 7f62a66d9640 1 -- 172.21.15.49:0/129436798 reap_dead start 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.795+0000 7f62a66d9640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f628c005180 0x7f628c00b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6298002230 tx=0x7f628c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679123d6354c9056 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.795+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629802cbb0 con 0x7f628c005180 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.795+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629802cd10 con 0x7f628c005180 2024-03-14T21:13:38.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.795+0000 7f62a4ed6640 1 -- 172.21.15.49:0/129436798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298035670 con 0x7f628c005180 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.796+0000 7f62a813c640 1 -- 172.21.15.49:0/129436798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f628c005180 msgr2=0x7f628c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.796+0000 7f62a813c640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f628c005180 0x7f628c00b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6298002230 tx=0x7f628c0057f0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.796+0000 7f62a813c640 1 -- 172.21.15.49:0/129436798 shutdown_connections 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.796+0000 7f62a813c640 1 --2- 172.21.15.49:0/129436798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f628c005180 0x7f628c00b980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.796+0000 7f62a813c640 1 -- 172.21.15.49:0/129436798 >> 172.21.15.49:0/129436798 conn(0x7f62a00b9d30 msgr2=0x7f62a00ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.796+0000 7f62a813c640 1 -- 172.21.15.49:0/129436798 shutdown_connections 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.796+0000 7f62a813c640 1 -- 172.21.15.49:0/129436798 wait complete. 2024-03-14T21:13:38.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.797+0000 7f62a813c640 1 Processor -- start 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.797+0000 7f62a813c640 1 -- start start 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a813c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c005180 0x7f62a00b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a813c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a01395d0 0x7f62a01430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a813c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62a0139a50 0x7f62a01435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a813c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00b2f90 con 0x7f62a01395d0 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a813c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0136c40 con 0x7f62a0139a50 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a813c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0144b40 con 0x7f628c005180 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a01395d0 0x7f62a01430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a01395d0 0x7f62a01430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39614/0 (socket says 172.21.15.49:39614) 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 -- 172.21.15.49:0/2629384382 learned_addr learned my addr 172.21.15.49:0/2629384382 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a5ed8640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c005180 0x7f62a00b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a66d9640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62a0139a50 0x7f62a01435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 -- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c005180 msgr2=0x7f62a00b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c005180 0x7f62a00b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 -- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62a0139a50 msgr2=0x7f62a01435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62a0139a50 0x7f62a01435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 -- 172.21.15.49:0/2629384382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a0144e20 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.798+0000 7f62a56d7640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a01395d0 0x7f62a01430a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f629c000c00 tx=0x7f629c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.799+0000 7f62967fc640 1 -- 172.21.15.49:0/2629384382 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0052e0 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.799+0000 7f62967fc640 1 -- 172.21.15.49:0/2629384382 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c005440 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.799+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a00b7210 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.799+0000 7f62967fc640 1 -- 172.21.15.49:0/2629384382 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00d970 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.799+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a0140640 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.799+0000 7f62967fc640 1 -- 172.21.15.49:0/2629384382 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f629c0035f0 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.799+0000 7f62967fc640 1 -- 172.21.15.49:0/2629384382 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f629c01d550 con 0x7f62a01395d0 2024-03-14T21:13:38.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.800+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a00b25c0 con 0x7f62a01395d0 2024-03-14T21:13:38.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.802+0000 7f62967fc640 1 -- 172.21.15.49:0/2629384382 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f62a00b25c0 con 0x7f62a01395d0 2024-03-14T21:13:38.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.868+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f62a0144970 con 0x7f62a01395d0 2024-03-14T21:13:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.870+0000 7f62967fc640 1 -- 172.21.15.49:0/2629384382 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f629c003a80 con 0x7f62a01395d0 2024-03-14T21:13:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a01395d0 msgr2=0x7f62a01430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a01395d0 0x7f62a01430a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f629c000c00 tx=0x7f629c00fd10 comp rx=0 tx=0).stop 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 shutdown_connections 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62a0139a50 0x7f62a01435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62a01395d0 0x7f62a01430a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 --2- 172.21.15.49:0/2629384382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f628c005180 0x7f62a00b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 >> 172.21.15.49:0/2629384382 conn(0x7f62a00b9d30 msgr2=0x7f62a00a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 shutdown_connections 2024-03-14T21:13:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:38.871+0000 7f62a813c640 1 -- 172.21.15.49:0/2629384382 wait complete. 2024-03-14T21:13:38.865 INFO:tasks.ceph:check_enable_crimson: False 2024-03-14T21:13:38.865 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-14T21:13:38.866 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:38.866 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-14T21:13:38.896 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:38.896 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-14T21:13:38.967 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:38.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-14T21:13:39.037 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-14T21:13:39.037 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-14T21:13:39.108 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:39.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-14T21:13:39.135 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:39.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-14T21:13:39.202 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:39.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-14T21:13:39.225 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:39.241+0000 7f68d8e21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-14T21:13:39.246 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:39.262+0000 7feb37a1e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-14T21:13:39.251 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-14T21:13:39.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-14T21:13:39.282 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:39.298+0000 7f68d8e21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-14T21:13:39.301 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:39.318+0000 7feb37a1e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-14T21:13:39.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new d49ebb7b-cd2e-4498-9607-a6bbdcefdb51 0 2024-03-14T21:13:39.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.463+0000 7fecdb8c4640 1 Processor -- start 2024-03-14T21:13:39.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecdb8c4640 1 -- start start 2024-03-14T21:13:39.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecdb8c4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd4145e30 0x7fecd406d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecdb8c4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fecd40ba450 0x7fecd406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecdb8c4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40b3500 con 0x7fecd406eb70 2024-03-14T21:13:39.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecdb8c4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd4146210 con 0x7fecd40ba450 2024-03-14T21:13:39.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecdb8c4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40b5460 con 0x7fecd4145e30 2024-03-14T21:13:39.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecd9660640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd4145e30 0x7fecd406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecd9660640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd4145e30 0x7fecd406d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:46894/0 (socket says 172.21.15.97:46894) 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.464+0000 7fecd9660640 1 -- 172.21.15.97:0/1575328427 learned_addr learned my addr 172.21.15.97:0/1575328427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.465+0000 7fecd8e5f640 1 --2- 172.21.15.97:0/1575328427 >> v2:172.21.15.49:3300/0 conn(0x7fecd40ba450 0x7fecd406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.465+0000 7fecd9660640 1 -- 172.21.15.97:0/1575328427 >> v2:172.21.15.49:3300/0 conn(0x7fecd40ba450 msgr2=0x7fecd406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.465+0000 7fecd9660640 1 --2- 172.21.15.97:0/1575328427 >> v2:172.21.15.49:3300/0 conn(0x7fecd40ba450 0x7fecd406e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.465+0000 7fecd9660640 1 -- 172.21.15.97:0/1575328427 >> v1:172.21.15.49:6789/0 conn(0x7fecd406eb70 legacy=0x7fecd414bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.465+0000 7fecd9660640 1 -- 172.21.15.97:0/1575328427 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40b3990 con 0x7fecd4145e30 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.465+0000 7fecd9660640 1 --2- 172.21.15.97:0/1575328427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd4145e30 0x7fecd406d970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fecd40b3b80 tx=0x7fecc402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a714a937de93091d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7feccbfff640 1 -- 172.21.15.97:0/1575328427 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4031640 con 0x7fecd4145e30 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7feccbfff640 1 -- 172.21.15.97:0/1575328427 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc40317a0 con 0x7fecd4145e30 2024-03-14T21:13:39.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7feccbfff640 1 -- 172.21.15.97:0/1575328427 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4031b30 con 0x7fecd4145e30 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1575328427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd4145e30 msgr2=0x7fecd406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 --2- 172.21.15.97:0/1575328427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd4145e30 0x7fecd406d970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fecd40b3b80 tx=0x7fecc402b8f0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1575328427 shutdown_connections 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 --2- 172.21.15.97:0/1575328427 >> v2:172.21.15.49:3300/0 conn(0x7fecd40ba450 0x7fecd406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 --2- 172.21.15.97:0/1575328427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd4145e30 0x7fecd406d970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1575328427 >> 172.21.15.97:0/1575328427 conn(0x7fecd40bad20 msgr2=0x7fecd40bb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1575328427 shutdown_connections 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.466+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1575328427 wait complete. 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.467+0000 7fecdb8c4640 1 Processor -- start 2024-03-14T21:13:39.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.467+0000 7fecdb8c4640 1 -- start start 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecdb8c4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd406eb70 0x7fecd413a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecdb8c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd40ba450 0x7fecd413acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecdb8c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecd4145e30 0x7fecd413cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecdb8c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd41396b0 con 0x7fecd4145e30 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecdb8c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd41399f0 con 0x7fecd40ba450 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecdb8c4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4139c90 con 0x7fecd406eb70 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd406eb70 0x7fecd413a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd406eb70 0x7fecd413a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:46900/0 (socket says 172.21.15.97:46900) 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 -- 172.21.15.97:0/1864667088 learned_addr learned my addr 172.21.15.97:0/1864667088 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd8e5f640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd40ba450 0x7fecd413acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9e61640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecd4145e30 0x7fecd413cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 -- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd40ba450 msgr2=0x7fecd413acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd40ba450 0x7fecd413acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 -- 172.21.15.97:0/1864667088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecd4145e30 msgr2=0x7fecd413cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecd4145e30 0x7fecd413cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 -- 172.21.15.97:0/1864667088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc40094d0 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9e61640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecd4145e30 0x7fecd413cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.468+0000 7fecd9660640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd406eb70 0x7fecd413a770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fecc402b930 tx=0x7fecc40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.469+0000 7fecc9ffb640 1 -- 172.21.15.97:0/1864667088 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc403a930 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.469+0000 7fecc9ffb640 1 -- 172.21.15.97:0/1864667088 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc403aa90 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.469+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd4139f70 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.469+0000 7fecc9ffb640 1 -- 172.21.15.97:0/1864667088 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc403ae40 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.469+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd40bde70 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.469+0000 7fecc9ffb640 1 -- 172.21.15.97:0/1864667088 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fecc402f050 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.469+0000 7fecc9ffb640 1 -- 172.21.15.97:0/1864667088 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fecc40444e0 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.470+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd413a490 con 0x7fecd406eb70 2024-03-14T21:13:39.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.472+0000 7fecc9ffb640 1 -- 172.21.15.97:0/1864667088 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fecc4035020 con 0x7fecd406eb70 2024-03-14T21:13:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.542+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d49ebb7b-cd2e-4498-9607-a6bbdcefdb51", "id": 0} v 0) v1 -- 0x7fecd406de30 con 0x7fecd406eb70 2024-03-14T21:13:39.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.545+0000 7fecc9ffb640 1 -- 172.21.15.97:0/1864667088 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d49ebb7b-cd2e-4498-9607-a6bbdcefdb51", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fecc402d040 con 0x7fecd406eb70 2024-03-14T21:13:39.528 INFO:teuthology.orchestra.run.smithi097.stdout:0 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd406eb70 msgr2=0x7fecd413a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd406eb70 0x7fecd413a770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fecc402b930 tx=0x7fecc40046c0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 shutdown_connections 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fecd4145e30 0x7fecd413cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecd40ba450 0x7fecd413acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 --2- 172.21.15.97:0/1864667088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fecd406eb70 0x7fecd413a770 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 >> 172.21.15.97:0/1864667088 conn(0x7fecd40bad20 msgr2=0x7fecd4144920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 shutdown_connections 2024-03-14T21:13:39.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.546+0000 7fecdb8c4640 1 -- 172.21.15.97:0/1864667088 wait complete. 2024-03-14T21:13:39.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 07801ea6-7081-49cf-ab8b-cfcb4a9c8113 1 2024-03-14T21:13:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.658+0000 7f8dcb9c9640 1 Processor -- start 2024-03-14T21:13:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.658+0000 7f8dcb9c9640 1 -- start start 2024-03-14T21:13:39.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dcb9c9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8dc406cf10 0x7f8dc4147c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dcb9c9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dc40b9930 0x7f8dc4148180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dcb9c9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc406d670 con 0x7f8dc40716f0 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dcb9c9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc40b8c40 con 0x7f8dc406cf10 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dcb9c9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc40b8f20 con 0x7f8dc40b9930 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dc8f64640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dc40b9930 0x7f8dc4148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dc8f64640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8dc40b9930 0x7f8dc4148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34124/0 (socket says 172.21.15.97:34124) 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.659+0000 7f8dc8f64640 1 -- 172.21.15.97:0/3887863493 learned_addr learned my addr 172.21.15.97:0/3887863493 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8dc9765640 1 --2- 172.21.15.97:0/3887863493 >> v2:172.21.15.49:3300/0 conn(0x7f8dc406cf10 0x7f8dc4147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8dc8f64640 1 -- 172.21.15.97:0/3887863493 >> v2:172.21.15.49:3300/0 conn(0x7f8dc406cf10 msgr2=0x7f8dc4147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8dc8f64640 1 --2- 172.21.15.97:0/3887863493 >> v2:172.21.15.49:3300/0 conn(0x7f8dc406cf10 0x7f8dc4147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8dc8f64640 1 -- 172.21.15.97:0/3887863493 >> v1:172.21.15.49:6789/0 conn(0x7f8dc40716f0 legacy=0x7f8dc41415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8dc8f64640 1 -- 172.21.15.97:0/3887863493 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc40b9200 con 0x7f8dc40b9930 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8dc9765640 1 --2- 172.21.15.97:0/3887863493 >> v2:172.21.15.49:3300/0 conn(0x7f8dc406cf10 0x7f8dc4147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8dc8f64640 1 --2- 172.21.15.97:0/3887863493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc4148180 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4148920 tx=0x7f8dc002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621107a40fd6ce3e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8db7fff640 1 -- 172.21.15.97:0/3887863493 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0031870 con 0x7f8dc40b9930 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8db7fff640 1 -- 172.21.15.97:0/3887863493 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc00319d0 con 0x7f8dc40b9930 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.660+0000 7f8db7fff640 1 -- 172.21.15.97:0/3887863493 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0031d60 con 0x7f8dc40b9930 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/3887863493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 msgr2=0x7f8dc4148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 --2- 172.21.15.97:0/3887863493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc4148180 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4148920 tx=0x7f8dc002b910 comp rx=0 tx=0).stop 2024-03-14T21:13:39.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/3887863493 shutdown_connections 2024-03-14T21:13:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 --2- 172.21.15.97:0/3887863493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc4148180 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 --2- 172.21.15.97:0/3887863493 >> v2:172.21.15.49:3300/0 conn(0x7f8dc406cf10 0x7f8dc4147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/3887863493 >> 172.21.15.97:0/3887863493 conn(0x7f8dc40b9d30 msgr2=0x7f8dc40ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/3887863493 shutdown_connections 2024-03-14T21:13:39.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.661+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/3887863493 wait complete. 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.662+0000 7f8dcb9c9640 1 Processor -- start 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.662+0000 7f8dcb9c9640 1 -- start start 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dcb9c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc406cf10 0x7f8dc40b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dcb9c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc40716f0 0x7f8dc41432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dcb9c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc413c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dcb9c9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4144280 con 0x7f8dc406cf10 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dcb9c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4142a10 con 0x7f8dc40b9930 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dcb9c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4142cb0 con 0x7f8dc40716f0 2024-03-14T21:13:39.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9f66640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc413c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc8f64640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc40716f0 0x7f8dc41432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc8f64640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc40716f0 0x7f8dc41432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:46914/0 (socket says 172.21.15.97:46914) 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc8f64640 1 -- 172.21.15.97:0/1503448585 learned_addr learned my addr 172.21.15.97:0/1503448585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9765640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc406cf10 0x7f8dc40b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9f66640 1 -- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc40716f0 msgr2=0x7f8dc41432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9f66640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc40716f0 0x7f8dc41432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9f66640 1 -- 172.21.15.97:0/1503448585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc406cf10 msgr2=0x7f8dc40b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9f66640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc406cf10 0x7f8dc40b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9f66640 1 -- 172.21.15.97:0/1503448585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00094d0 con 0x7f8dc40b9930 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc8f64640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc40716f0 0x7f8dc41432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.663+0000 7f8dc9f66640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc413c4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00c780 tx=0x7f8dbc00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:39.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.664+0000 7f8db5ffb640 1 -- 172.21.15.97:0/1503448585 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00aa50 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.664+0000 7f8db5ffb640 1 -- 172.21.15.97:0/1503448585 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc00abb0 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.664+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc40b6da0 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.664+0000 7f8db5ffb640 1 -- 172.21.15.97:0/1503448585 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00ce30 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.664+0000 7f8db5ffb640 1 -- 172.21.15.97:0/1503448585 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8dbc012720 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.664+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc413f200 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.664+0000 7f8db5ffb640 1 -- 172.21.15.97:0/1503448585 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f8dbc01b400 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.665+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc4073540 con 0x7f8dc40b9930 2024-03-14T21:13:39.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.667+0000 7f8db5ffb640 1 -- 172.21.15.97:0/1503448585 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f8dbc03ab50 con 0x7f8dc40b9930 2024-03-14T21:13:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.738+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "07801ea6-7081-49cf-ab8b-cfcb4a9c8113", "id": 1} v 0) v1 -- 0x7f8dc4142df0 con 0x7f8dc40b9930 2024-03-14T21:13:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.741+0000 7f8db5ffb640 1 -- 172.21.15.97:0/1503448585 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07801ea6-7081-49cf-ab8b-cfcb4a9c8113", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8dbc029070 con 0x7f8dc40b9930 2024-03-14T21:13:39.725 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-03-14T21:13:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 msgr2=0x7f8dc413c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc413c4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00c780 tx=0x7f8dbc00cc40 comp rx=0 tx=0).stop 2024-03-14T21:13:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 shutdown_connections 2024-03-14T21:13:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc40b9930 0x7f8dc413c4d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc40716f0 0x7f8dc41432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 --2- 172.21.15.97:0/1503448585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc406cf10 0x7f8dc40b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 >> 172.21.15.97:0/1503448585 conn(0x7f8dc40b9d30 msgr2=0x7f8dc41415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 shutdown_connections 2024-03-14T21:13:39.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.742+0000 7f8dcb9c9640 1 -- 172.21.15.97:0/1503448585 wait complete. 2024-03-14T21:13:39.736 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 698263f6-bc7c-41a8-aa99-be46814ddb8d 2 2024-03-14T21:13:39.820 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:39.836+0000 7f68d8e21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-14T21:13:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.852+0000 7f35d2026640 1 Processor -- start 2024-03-14T21:13:39.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.852+0000 7f35d2026640 1 -- start start 2024-03-14T21:13:39.837 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:39.853+0000 7feb37a1e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-14T21:13:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35d2026640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35cc0bbed0 0x7f35cc14e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35d2026640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f35cc0bc340 0x7f35cc14e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35d2026640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35cc0b86a0 con 0x7f35cc0b7e80 2024-03-14T21:13:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35d2026640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc09bf60 con 0x7f35cc0bbed0 2024-03-14T21:13:39.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35d2026640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc09c240 con 0x7f35cc0bc340 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35cb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f35cc0b7e80 0x7f35cc071dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38890/0 (socket says 172.21.15.97:38890) 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35cb7fe640 1 -- 172.21.15.97:0/1444799923 learned_addr learned my addr 172.21.15.97:0/1444799923 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35cbfff640 1 --2- 172.21.15.97:0/1444799923 >> v2:172.21.15.97:3300/0 conn(0x7f35cc0bc340 0x7f35cc14e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35c2ffd640 1 --2- 172.21.15.97:0/1444799923 >> v2:172.21.15.49:3300/0 conn(0x7f35cc0bbed0 0x7f35cc14e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35caffd640 1 -- 172.21.15.97:0/1444799923 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238556666 0 0) 0x7f35cc0b86a0 con 0x7f35cc0b7e80 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35caffd640 1 -- 172.21.15.97:0/1444799923 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35b8003520 con 0x7f35cc0b7e80 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35cbfff640 1 -- 172.21.15.97:0/1444799923 >> v2:172.21.15.49:3300/0 conn(0x7f35cc0bbed0 msgr2=0x7f35cc14e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35cbfff640 1 --2- 172.21.15.97:0/1444799923 >> v2:172.21.15.49:3300/0 conn(0x7f35cc0bbed0 0x7f35cc14e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35cbfff640 1 -- 172.21.15.97:0/1444799923 >> v1:172.21.15.97:6789/0 conn(0x7f35cc0b7e80 legacy=0x7f35cc071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.853+0000 7f35cbfff640 1 -- 172.21.15.97:0/1444799923 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc0bb760 con 0x7f35cc0bc340 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.854+0000 7f35cbfff640 1 --2- 172.21.15.97:0/1444799923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc0bc340 0x7f35cc14e8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009d90 tx=0x7f35c402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569262146e8236d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:39.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.854+0000 7f35caffd640 1 -- 172.21.15.97:0/1444799923 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c402cbd0 con 0x7f35cc0bc340 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.854+0000 7f35caffd640 1 -- 172.21.15.97:0/1444799923 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c402cd30 con 0x7f35cc0bc340 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.854+0000 7f35caffd640 1 -- 172.21.15.97:0/1444799923 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c402cea0 con 0x7f35cc0bc340 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 -- 172.21.15.97:0/1444799923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc0bc340 msgr2=0x7f35cc14e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 --2- 172.21.15.97:0/1444799923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc0bc340 0x7f35cc14e8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009d90 tx=0x7f35c402be20 comp rx=0 tx=0).stop 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 -- 172.21.15.97:0/1444799923 shutdown_connections 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 --2- 172.21.15.97:0/1444799923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc0bc340 0x7f35cc14e8f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 --2- 172.21.15.97:0/1444799923 >> v2:172.21.15.49:3300/0 conn(0x7f35cc0bbed0 0x7f35cc14e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 -- 172.21.15.97:0/1444799923 >> 172.21.15.97:0/1444799923 conn(0x7f35cc0b93b0 msgr2=0x7f35cc0b97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 -- 172.21.15.97:0/1444799923 shutdown_connections 2024-03-14T21:13:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.855+0000 7f35d2026640 1 -- 172.21.15.97:0/1444799923 wait complete. 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.856+0000 7f35d2026640 1 Processor -- start 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.856+0000 7f35d2026640 1 -- start start 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35d2026640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35cc0b7e80 0x7f35cc0a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35d2026640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc1491e0 0x7f35cc1dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35d2026640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35cc144db0 0x7f35cc145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35d2026640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0a0d60 con 0x7f35cc0b7e80 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35d2026640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc149aa0 con 0x7f35cc1491e0 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35d2026640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc14a000 con 0x7f35cc144db0 2024-03-14T21:13:39.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc1491e0 0x7f35cc1dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35cbfff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35cc144db0 0x7f35cc145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc1491e0 0x7f35cc1dfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34154/0 (socket says 172.21.15.97:34154) 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 -- 172.21.15.97:0/4277131086 learned_addr learned my addr 172.21.15.97:0/4277131086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35cb7fe640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35cc0b7e80 0x7f35cc0a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 -- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35cc144db0 msgr2=0x7f35cc145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35cc144db0 0x7f35cc145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 -- 172.21.15.97:0/4277131086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35cc0b7e80 msgr2=0x7f35cc0a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35cc0b7e80 0x7f35cc0a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 -- 172.21.15.97:0/4277131086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c4009a70 con 0x7f35cc1491e0 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35cbfff640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35cc144db0 0x7f35cc145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.857+0000 7f35c2ffd640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc1491e0 0x7f35cc1dfce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f35b400e980 tx=0x7f35b400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.858+0000 7f35c8ff9640 1 -- 172.21.15.97:0/4277131086 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4009670 con 0x7f35cc1491e0 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.858+0000 7f35c8ff9640 1 -- 172.21.15.97:0/4277131086 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b400ca00 con 0x7f35cc1491e0 2024-03-14T21:13:39.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.858+0000 7f35c8ff9640 1 -- 172.21.15.97:0/4277131086 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400cd90 con 0x7f35cc1491e0 2024-03-14T21:13:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.858+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc14aba0 con 0x7f35cc1491e0 2024-03-14T21:13:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.858+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc149cd0 con 0x7f35cc1491e0 2024-03-14T21:13:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.858+0000 7f35c8ff9640 1 -- 172.21.15.97:0/4277131086 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f35b4005c90 con 0x7f35cc1491e0 2024-03-14T21:13:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.858+0000 7f35c8ff9640 1 -- 172.21.15.97:0/4277131086 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f35b4013890 con 0x7f35cc1491e0 2024-03-14T21:13:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.859+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc09f750 con 0x7f35cc1491e0 2024-03-14T21:13:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.863+0000 7f35c8ff9640 1 -- 172.21.15.97:0/4277131086 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f35b4021060 con 0x7f35cc1491e0 2024-03-14T21:13:39.876 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:39.893+0000 7f68d8e21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-14T21:13:39.891 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:39.908+0000 7feb37a1e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-14T21:13:39.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.934+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "698263f6-bc7c-41a8-aa99-be46814ddb8d", "id": 2} v 0) v1 -- 0x7f35cc073540 con 0x7f35cc1491e0 2024-03-14T21:13:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.937+0000 7f35c8ff9640 1 -- 172.21.15.97:0/4277131086 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "698263f6-bc7c-41a8-aa99-be46814ddb8d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f35b4017070 con 0x7f35cc1491e0 2024-03-14T21:13:39.921 INFO:teuthology.orchestra.run.smithi097.stdout:2 2024-03-14T21:13:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc1491e0 msgr2=0x7f35cc1dfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc1491e0 0x7f35cc1dfce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f35b400e980 tx=0x7f35b400ee40 comp rx=0 tx=0).stop 2024-03-14T21:13:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 shutdown_connections 2024-03-14T21:13:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f35cc144db0 0x7f35cc145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35cc1491e0 0x7f35cc1dfce0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 --2- 172.21.15.97:0/4277131086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35cc0b7e80 0x7f35cc0a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 >> 172.21.15.97:0/4277131086 conn(0x7f35cc0b93b0 msgr2=0x7f35cc14b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 shutdown_connections 2024-03-14T21:13:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:39.938+0000 7f35d2026640 1 -- 172.21.15.97:0/4277131086 wait complete. 2024-03-14T21:13:39.923 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:39.940+0000 7f68d8e21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-14T21:13:39.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new c551f38b-c910-4d22-b9de-5e8fc2356379 3 2024-03-14T21:13:39.937 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:39.954+0000 7feb37a1e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-14T21:13:39.980 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:39.997+0000 7f68d8e21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-14T21:13:39.993 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.010+0000 7feb37a1e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-14T21:13:40.024 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.041+0000 7f68d8e21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-14T21:13:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.047+0000 7f177fa5c640 1 Processor -- start 2024-03-14T21:13:40.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.047+0000 7f177fa5c640 1 -- start start 2024-03-14T21:13:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177fa5c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17780bc8c0 0x7f177806a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177fa5c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f177806ae50 0x7f1778155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177fa5c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17780bb310 con 0x7f177814b710 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177fa5c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17780bbab0 con 0x7f177806ae50 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177fa5c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f177814aae0 con 0x7f17780bc8c0 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177cff7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17780bc8c0 0x7f177806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177cff7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17780bc8c0 0x7f177806a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34170/0 (socket says 172.21.15.97:34170) 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177cff7640 1 -- 172.21.15.97:0/2562665032 learned_addr learned my addr 172.21.15.97:0/2562665032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177dff9640 1 --2- 172.21.15.97:0/2562665032 >> v2:172.21.15.49:3300/0 conn(0x7f177806ae50 0x7f1778155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177cff7640 1 -- 172.21.15.97:0/2562665032 >> v2:172.21.15.49:3300/0 conn(0x7f177806ae50 msgr2=0x7f1778155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177cff7640 1 --2- 172.21.15.97:0/2562665032 >> v2:172.21.15.49:3300/0 conn(0x7f177806ae50 0x7f1778155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177cff7640 1 -- 172.21.15.97:0/2562665032 >> v1:172.21.15.49:6789/0 conn(0x7f177814b710 legacy=0x7f17780bac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.048+0000 7f177cff7640 1 -- 172.21.15.97:0/2562665032 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17780bfa80 con 0x7f17780bc8c0 2024-03-14T21:13:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.049+0000 7f177cff7640 1 --2- 172.21.15.97:0/2562665032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f177806a910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1778061a20 tx=0x7f177402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71763ad4cc49bd33 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.049+0000 7f176bfff640 1 -- 172.21.15.97:0/2562665032 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774030860 con 0x7f17780bc8c0 2024-03-14T21:13:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.049+0000 7f176bfff640 1 -- 172.21.15.97:0/2562665032 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17740309c0 con 0x7f17780bc8c0 2024-03-14T21:13:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.049+0000 7f176bfff640 1 -- 172.21.15.97:0/2562665032 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774030d50 con 0x7f17780bc8c0 2024-03-14T21:13:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 -- 172.21.15.97:0/2562665032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 msgr2=0x7f177806a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 --2- 172.21.15.97:0/2562665032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f177806a910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1778061a20 tx=0x7f177402b9a0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 -- 172.21.15.97:0/2562665032 shutdown_connections 2024-03-14T21:13:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 --2- 172.21.15.97:0/2562665032 >> v2:172.21.15.49:3300/0 conn(0x7f177806ae50 0x7f1778155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 --2- 172.21.15.97:0/2562665032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f177806a910 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 -- 172.21.15.97:0/2562665032 >> 172.21.15.97:0/2562665032 conn(0x7f17780c0460 msgr2=0x7f17780c0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 -- 172.21.15.97:0/2562665032 shutdown_connections 2024-03-14T21:13:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.050+0000 7f177fa5c640 1 -- 172.21.15.97:0/2562665032 wait complete. 2024-03-14T21:13:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.051+0000 7f177fa5c640 1 Processor -- start 2024-03-14T21:13:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.052+0000 7f177fa5c640 1 -- start start 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.052+0000 7f177fa5c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f177806ae50 0x7f17780a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.052+0000 7f177fa5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f17781dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.052+0000 7f177fa5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177814b710 0x7f17780b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.052+0000 7f177fa5c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17780a5b50 con 0x7f177814b710 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.052+0000 7f177fa5c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17781495b0 con 0x7f17780bc8c0 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177fa5c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17780b3580 con 0x7f177806ae50 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f17781dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177d7f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f177806ae50 0x7f17780a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f17781dffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34178/0 (socket says 172.21.15.97:34178) 2024-03-14T21:13:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 -- 172.21.15.97:0/1296716394 learned_addr learned my addr 172.21.15.97:0/1296716394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177dff9640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177814b710 0x7f17780b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 -- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f177806ae50 msgr2=0x7f17780a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f177806ae50 0x7f17780a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 -- 172.21.15.97:0/1296716394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177814b710 msgr2=0x7f17780b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177814b710 0x7f17780b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 -- 172.21.15.97:0/1296716394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740094d0 con 0x7f17780bc8c0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177d7f8640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f177806ae50 0x7f17780a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177cff7640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f17781dffd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f177402b9e0 tx=0x7f1774004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f1769ffb640 1 -- 172.21.15.97:0/1296716394 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774039d60 con 0x7f17780bc8c0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f1769ffb640 1 -- 172.21.15.97:0/1296716394 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1774043400 con 0x7f17780bc8c0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f1769ffb640 1 -- 172.21.15.97:0/1296716394 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740437b0 con 0x7f17780bc8c0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.053+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17780b3790 con 0x7f17780bc8c0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.054+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17781483e0 con 0x7f17780bc8c0 2024-03-14T21:13:40.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.054+0000 7f1769ffb640 1 -- 172.21.15.97:0/1296716394 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1774043c10 con 0x7f17780bc8c0 2024-03-14T21:13:40.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.054+0000 7f1769ffb640 1 -- 172.21.15.97:0/1296716394 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f177403e070 con 0x7f17780bc8c0 2024-03-14T21:13:40.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.054+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1778148570 con 0x7f17780bc8c0 2024-03-14T21:13:40.041 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.054+0000 7feb37a1e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-14T21:13:40.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.056+0000 7f1769ffb640 1 -- 172.21.15.97:0/1296716394 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f1778148570 con 0x7f17780bc8c0 2024-03-14T21:13:40.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.128+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c551f38b-c910-4d22-b9de-5e8fc2356379", "id": 3} v 0) v1 -- 0x7f17780b3920 con 0x7f17780bc8c0 2024-03-14T21:13:40.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.131+0000 7f1769ffb640 1 -- 172.21.15.97:0/1296716394 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c551f38b-c910-4d22-b9de-5e8fc2356379", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1774034070 con 0x7f17780bc8c0 2024-03-14T21:13:40.115 INFO:teuthology.orchestra.run.smithi097.stdout:3 2024-03-14T21:13:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 msgr2=0x7f17781dffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f17781dffd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f177402b9e0 tx=0x7f1774004730 comp rx=0 tx=0).stop 2024-03-14T21:13:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 shutdown_connections 2024-03-14T21:13:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f177814b710 0x7f17780b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17780bc8c0 0x7f17781dffd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 --2- 172.21.15.97:0/1296716394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f177806ae50 0x7f17780a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 >> 172.21.15.97:0/1296716394 conn(0x7f17780c0460 msgr2=0x7f17780b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 shutdown_connections 2024-03-14T21:13:40.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.132+0000 7f177fa5c640 1 -- 172.21.15.97:0/1296716394 wait complete. 2024-03-14T21:13:40.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new cd43c8e4-a723-4ed4-8c61-dea1188c1b35 4 2024-03-14T21:13:40.146 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.163+0000 7f68d8e21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-14T21:13:40.158 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.175+0000 7feb37a1e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-14T21:13:40.220 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.236+0000 7f68d8e21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-14T21:13:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d0aeea640 1 Processor -- start 2024-03-14T21:13:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d0aeea640 1 -- start start 2024-03-14T21:13:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d0aeea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d04148360 0x7f3d0406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d0aeea640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d04147ed0 con 0x7f3d0406d170 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d0aeea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d04147880 con 0x7f3d04141ae0 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d0aeea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0406d9c0 con 0x7f3d04148360 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d03fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d04148360 0x7f3d0406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d08c86640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3d04141ae0 0x7f3d0406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38902/0 (socket says 172.21.15.97:38902) 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.242+0000 7f3d08c86640 1 -- 172.21.15.97:0/2794388958 learned_addr learned my addr 172.21.15.97:0/2794388958 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.243+0000 7f3d03fff640 1 -- 172.21.15.97:0/2794388958 >> v1:172.21.15.97:6789/0 conn(0x7f3d04141ae0 legacy=0x7f3d0406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.243+0000 7f3d03fff640 1 -- 172.21.15.97:0/2794388958 >> v1:172.21.15.49:6789/0 conn(0x7f3d0406d170 legacy=0x7f3d04151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.243+0000 7f3d037fe640 1 -- 172.21.15.97:0/2794388958 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409223037 0 0) 0x7f3d04147880 con 0x7f3d04141ae0 2024-03-14T21:13:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.243+0000 7f3d03fff640 1 -- 172.21.15.97:0/2794388958 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d04147b60 con 0x7f3d04148360 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.243+0000 7f3d03fff640 1 --2- 172.21.15.97:0/2794388958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d04148360 0x7f3d0406cc30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00097d0 tx=0x7f3cf002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0ba7c79c3ca6eb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.243+0000 7f3d037fe640 1 -- 172.21.15.97:0/2794388958 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0030610 con 0x7f3d04148360 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d037fe640 1 -- 172.21.15.97:0/2794388958 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf0030770 con 0x7f3d04148360 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d037fe640 1 -- 172.21.15.97:0/2794388958 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0030b00 con 0x7f3d04148360 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2794388958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d04148360 msgr2=0x7f3d0406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d0aeea640 1 --2- 172.21.15.97:0/2794388958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d04148360 0x7f3d0406cc30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00097d0 tx=0x7f3cf002b960 comp rx=0 tx=0).stop 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2794388958 shutdown_connections 2024-03-14T21:13:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d0aeea640 1 --2- 172.21.15.97:0/2794388958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d04148360 0x7f3d0406cc30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2794388958 >> 172.21.15.97:0/2794388958 conn(0x7f3d040b9d30 msgr2=0x7f3d040ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2794388958 shutdown_connections 2024-03-14T21:13:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.244+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2794388958 wait complete. 2024-03-14T21:13:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.245+0000 7f3d0aeea640 1 Processor -- start 2024-03-14T21:13:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.245+0000 7f3d0aeea640 1 -- start start 2024-03-14T21:13:40.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d0aeea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d04141ae0 0x7f3d040bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d0aeea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d04148360 0x7f3d0413b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d0aeea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0413bc30 0x7f3d0414ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d0aeea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040bcde0 con 0x7f3d04141ae0 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d0aeea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d04149f40 con 0x7f3d0413bc30 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d0aeea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040b5860 con 0x7f3d04148360 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0413bc30 0x7f3d0414ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d03fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d04148360 0x7f3d0413b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0413bc30 0x7f3d0414ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34188/0 (socket says 172.21.15.97:34188) 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d03fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d04148360 0x7f3d0413b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:46934/0 (socket says 172.21.15.97:46934) 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 -- 172.21.15.97:0/2940675343 learned_addr learned my addr 172.21.15.97:0/2940675343 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d08c86640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d04141ae0 0x7f3d040bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 -- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d04148360 msgr2=0x7f3d0413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d04148360 0x7f3d0413b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 -- 172.21.15.97:0/2940675343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d04141ae0 msgr2=0x7f3d040bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d04141ae0 0x7f3d040bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d09487640 1 -- 172.21.15.97:0/2940675343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0009400 con 0x7f3d0413bc30 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.246+0000 7f3d03fff640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d04148360 0x7f3d0413b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d09487640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0413bc30 0x7f3d0414ae10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c730 tx=0x7f3cfc00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d017fa640 1 -- 172.21.15.97:0/2940675343 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc003b70 con 0x7f3d0413bc30 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d017fa640 1 -- 172.21.15.97:0/2940675343 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc003cd0 con 0x7f3d0413bc30 2024-03-14T21:13:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0414a680 con 0x7f3d0413bc30 2024-03-14T21:13:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d017fa640 1 -- 172.21.15.97:0/2940675343 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00aa90 con 0x7f3d0413bc30 2024-03-14T21:13:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d04148c70 con 0x7f3d0413bc30 2024-03-14T21:13:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d017fa640 1 -- 172.21.15.97:0/2940675343 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3cfc0050d0 con 0x7f3d0413bc30 2024-03-14T21:13:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.247+0000 7f3d017fa640 1 -- 172.21.15.97:0/2940675343 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f3cfc012930 con 0x7f3d0413bc30 2024-03-14T21:13:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.248+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d04073540 con 0x7f3d0413bc30 2024-03-14T21:13:40.236 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.248+0000 7feb37a1e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-14T21:13:40.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.250+0000 7f3d017fa640 1 -- 172.21.15.97:0/2940675343 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f3cfc005560 con 0x7f3d0413bc30 2024-03-14T21:13:40.268 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.285+0000 7f68d8e21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-14T21:13:40.279 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.296+0000 7feb37a1e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-14T21:13:40.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.321+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cd43c8e4-a723-4ed4-8c61-dea1188c1b35", "id": 4} v 0) v1 -- 0x7f3d0406c1f0 con 0x7f3d0413bc30 2024-03-14T21:13:40.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.324+0000 7f3d017fa640 1 -- 172.21.15.97:0/2940675343 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cd43c8e4-a723-4ed4-8c61-dea1188c1b35", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3cfc005700 con 0x7f3d0413bc30 2024-03-14T21:13:40.308 INFO:teuthology.orchestra.run.smithi097.stdout:4 2024-03-14T21:13:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0413bc30 msgr2=0x7f3d0414ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0413bc30 0x7f3d0414ae10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c730 tx=0x7f3cfc00cbf0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 shutdown_connections 2024-03-14T21:13:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0413bc30 0x7f3d0414ae10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3d04148360 0x7f3d0413b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 --2- 172.21.15.97:0/2940675343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d04141ae0 0x7f3d040bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 >> 172.21.15.97:0/2940675343 conn(0x7f3d040b9d30 msgr2=0x7f3d0413f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 shutdown_connections 2024-03-14T21:13:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.325+0000 7f3d0aeea640 1 -- 172.21.15.97:0/2940675343 wait complete. 2024-03-14T21:13:40.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new d527b87c-b6bd-4a30-9c49-a0e640234a12 5 2024-03-14T21:13:40.323 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.340+0000 7f68d8e21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-14T21:13:40.333 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.350+0000 7feb37a1e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-14T21:13:40.373 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.389+0000 7f68d8e21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-14T21:13:40.382 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.399+0000 7feb37a1e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-14T21:13:40.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.438+0000 7f3228151640 1 Processor -- start 2024-03-14T21:13:40.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.438+0000 7f3228151640 1 -- start start 2024-03-14T21:13:40.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f3228151640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3220149280 0x7f32201540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f3228151640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322014c340 con 0x7f322006ffe0 2024-03-14T21:13:40.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f3228151640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322014c620 con 0x7f32200b8eb0 2024-03-14T21:13:40.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f3228151640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200b23e0 con 0x7f3220149280 2024-03-14T21:13:40.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f32266ee640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3220149280 0x7f32201540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f3225eed640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32200b8eb0 0x7f32201427f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38912/0 (socket says 172.21.15.97:38912) 2024-03-14T21:13:40.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f3225eed640 1 -- 172.21.15.97:0/957158861 learned_addr learned my addr 172.21.15.97:0/957158861 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-14T21:13:40.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f32266ee640 1 -- 172.21.15.97:0/957158861 >> v1:172.21.15.97:6789/0 conn(0x7f32200b8eb0 legacy=0x7f32201427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f32266ee640 1 -- 172.21.15.97:0/957158861 >> v1:172.21.15.49:6789/0 conn(0x7f322006ffe0 legacy=0x7f3220148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.439+0000 7f32266ee640 1 -- 172.21.15.97:0/957158861 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322014c900 con 0x7f3220149280 2024-03-14T21:13:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.440+0000 7f32266ee640 1 --2- 172.21.15.97:0/957158861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3220149280 0x7f32201540d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3218009d90 tx=0x7f321802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c612c8ee5e433a75 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.440+0000 7f3224eeb640 1 -- 172.21.15.97:0/957158861 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321802cd00 con 0x7f3220149280 2024-03-14T21:13:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.440+0000 7f3224eeb640 1 -- 172.21.15.97:0/957158861 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321802ce60 con 0x7f3220149280 2024-03-14T21:13:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.440+0000 7f3224eeb640 1 -- 172.21.15.97:0/957158861 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218034660 con 0x7f3220149280 2024-03-14T21:13:40.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.441+0000 7f3228151640 1 -- 172.21.15.97:0/957158861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3220149280 msgr2=0x7f32201540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.441+0000 7f3228151640 1 --2- 172.21.15.97:0/957158861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3220149280 0x7f32201540d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3218009d90 tx=0x7f321802bf50 comp rx=0 tx=0).stop 2024-03-14T21:13:40.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.441+0000 7f3228151640 1 -- 172.21.15.97:0/957158861 shutdown_connections 2024-03-14T21:13:40.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.441+0000 7f3228151640 1 --2- 172.21.15.97:0/957158861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3220149280 0x7f32201540d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.441+0000 7f3228151640 1 -- 172.21.15.97:0/957158861 >> 172.21.15.97:0/957158861 conn(0x7f32200b9d30 msgr2=0x7f32200ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.441+0000 7f3228151640 1 -- 172.21.15.97:0/957158861 shutdown_connections 2024-03-14T21:13:40.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.441+0000 7f3228151640 1 -- 172.21.15.97:0/957158861 wait complete. 2024-03-14T21:13:40.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.442+0000 7f3228151640 1 Processor -- start 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.442+0000 7f3228151640 1 -- start start 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f322006ffe0 0x7f32200a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32200b8eb0 0x7f32201e3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3220149280 0x7f32201e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200a2200 con 0x7f3220149280 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3220140940 con 0x7f32200b8eb0 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3220149ab0 con 0x7f322006ffe0 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32200b8eb0 0x7f32201e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3225eed640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f322006ffe0 0x7f32200a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32200b8eb0 0x7f32201e3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34216/0 (socket says 172.21.15.97:34216) 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 -- 172.21.15.97:0/2746865379 learned_addr learned my addr 172.21.15.97:0/2746865379 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32266ee640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3220149280 0x7f32201e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 -- 172.21.15.97:0/2746865379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f322006ffe0 msgr2=0x7f32200a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f322006ffe0 0x7f32200a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 -- 172.21.15.97:0/2746865379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3220149280 msgr2=0x7f32201e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3220149280 0x7f32201e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 -- 172.21.15.97:0/2746865379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3218009a70 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32256ec640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32200b8eb0 0x7f32201e3560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c450 tx=0x7f321c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32167fc640 1 -- 172.21.15.97:0/2746865379 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0052d0 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32167fc640 1 -- 172.21.15.97:0/2746865379 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c005430 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3220140bc0 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f32167fc640 1 -- 172.21.15.97:0/2746865379 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c012870 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.443+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3220141110 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.444+0000 7f32167fc640 1 -- 172.21.15.97:0/2746865379 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f321c012cd0 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.444+0000 7f32167fc640 1 -- 172.21.15.97:0/2746865379 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f321c01b980 con 0x7f32200b8eb0 2024-03-14T21:13:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.445+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32201e3280 con 0x7f32200b8eb0 2024-03-14T21:13:40.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.447+0000 7f32167fc640 1 -- 172.21.15.97:0/2746865379 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f32201e3280 con 0x7f32200b8eb0 2024-03-14T21:13:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.535+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d527b87c-b6bd-4a30-9c49-a0e640234a12", "id": 5} v 0) v1 -- 0x7f3220073540 con 0x7f32200b8eb0 2024-03-14T21:13:40.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.538+0000 7f32167fc640 1 -- 172.21.15.97:0/2746865379 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d527b87c-b6bd-4a30-9c49-a0e640234a12", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f321c02a0b0 con 0x7f32200b8eb0 2024-03-14T21:13:40.522 INFO:teuthology.orchestra.run.smithi097.stdout:5 2024-03-14T21:13:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32200b8eb0 msgr2=0x7f32201e3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32200b8eb0 0x7f32201e3560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c450 tx=0x7f321c00c910 comp rx=0 tx=0).stop 2024-03-14T21:13:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 shutdown_connections 2024-03-14T21:13:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3220149280 0x7f32201e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32200b8eb0 0x7f32201e3560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 --2- 172.21.15.97:0/2746865379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f322006ffe0 0x7f32200a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 >> 172.21.15.97:0/2746865379 conn(0x7f32200b9d30 msgr2=0x7f3220070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 shutdown_connections 2024-03-14T21:13:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.539+0000 7f3228151640 1 -- 172.21.15.97:0/2746865379 wait complete. 2024-03-14T21:13:40.533 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 17236019-5f1b-4e4f-b3ab-bb0b664551b1 6 2024-03-14T21:13:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.648+0000 7f96b3c84640 1 Processor -- start 2024-03-14T21:13:40.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.648+0000 7f96b3c84640 1 -- start start 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b3c84640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac148a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b3c84640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96ac0b9160 0x7f96ac148f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b3c84640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ac0b9dc0 con 0x7f96ac145e00 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b3c84640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ac071fb0 con 0x7f96ac0b9160 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b3c84640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0ba010 con 0x7f96ac0b8d20 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b121f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b2221640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96ac0b9160 0x7f96ac148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b121f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:46950/0 (socket says 172.21.15.97:46950) 2024-03-14T21:13:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b121f640 1 -- 172.21.15.97:0/1343066275 learned_addr learned my addr 172.21.15.97:0/1343066275 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b2221640 1 -- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 msgr2=0x7f96ac148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b2221640 1 --2- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.649+0000 7f96b2221640 1 -- 172.21.15.97:0/1343066275 >> v1:172.21.15.49:6789/0 conn(0x7f96ac145e00 legacy=0x7f96ac0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.650+0000 7f96b2221640 1 -- 172.21.15.97:0/1343066275 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac0ba2f0 con 0x7f96ac0b9160 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.650+0000 7f96b121f640 1 --2- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.650+0000 7f96b2221640 1 --2- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac148f60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009d90 tx=0x7f96a402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1e28b6cf8d86be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.650+0000 7f96b0a1e640 1 -- 172.21.15.97:0/1343066275 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a402cce0 con 0x7f96ac0b9160 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.650+0000 7f96b0a1e640 1 -- 172.21.15.97:0/1343066275 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a402ce40 con 0x7f96ac0b9160 2024-03-14T21:13:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.650+0000 7f96b0a1e640 1 -- 172.21.15.97:0/1343066275 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4034660 con 0x7f96ac0b9160 2024-03-14T21:13:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 -- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 msgr2=0x7f96ac148f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 --2- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac148f60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009d90 tx=0x7f96a402bf30 comp rx=0 tx=0).stop 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 -- 172.21.15.97:0/1343066275 shutdown_connections 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 --2- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 --2- 172.21.15.97:0/1343066275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac148f60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 -- 172.21.15.97:0/1343066275 >> 172.21.15.97:0/1343066275 conn(0x7f96ac06a310 msgr2=0x7f96ac071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 -- 172.21.15.97:0/1343066275 shutdown_connections 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.651+0000 7f96b3c84640 1 -- 172.21.15.97:0/1343066275 wait complete. 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.652+0000 7f96b3c84640 1 Processor -- start 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.652+0000 7f96b3c84640 1 -- start start 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b3c84640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac0be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b3c84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac1e1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b3c84640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96ac145e00 0x7f96ac0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b3c84640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0beb20 con 0x7f96ac145e00 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b3c84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0b82f0 con 0x7f96ac0b9160 2024-03-14T21:13:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b3c84640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac1e1bc0 con 0x7f96ac0b8d20 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b1a20640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac0be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b121f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac1e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b1a20640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac0be5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:46954/0 (socket says 172.21.15.97:46954) 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b1a20640 1 -- 172.21.15.97:0/4068633000 learned_addr learned my addr 172.21.15.97:0/4068633000 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b2221640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96ac145e00 0x7f96ac0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b121f640 1 -- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 msgr2=0x7f96ac0be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b121f640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac0be5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b121f640 1 -- 172.21.15.97:0/4068633000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96ac145e00 msgr2=0x7f96ac0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b121f640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96ac145e00 0x7f96ac0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b121f640 1 -- 172.21.15.97:0/4068633000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a8009460 con 0x7f96ac0b9160 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b1a20640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac0be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b2221640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96ac145e00 0x7f96ac0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b121f640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac1e1650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f96a8010d10 tx=0x7f96a800b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96a27fc640 1 -- 172.21.15.97:0/4068633000 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8015070 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96a27fc640 1 -- 172.21.15.97:0/4068633000 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a800bdd0 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.653+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a4009a70 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.654+0000 7f96a27fc640 1 -- 172.21.15.97:0/4068633000 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8005bb0 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.654+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac1de540 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.654+0000 7f96a27fc640 1 -- 172.21.15.97:0/4068633000 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f96a8006330 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.654+0000 7f96a27fc640 1 -- 172.21.15.97:0/4068633000 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f96a8019d10 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.654+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac073540 con 0x7f96ac0b9160 2024-03-14T21:13:40.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.656+0000 7f96a27fc640 1 -- 172.21.15.97:0/4068633000 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f96a8027060 con 0x7f96ac0b9160 2024-03-14T21:13:40.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.728+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "17236019-5f1b-4e4f-b3ab-bb0b664551b1", "id": 6} v 0) v1 -- 0x7f96ac0a0fa0 con 0x7f96ac0b9160 2024-03-14T21:13:40.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.731+0000 7f96a27fc640 1 -- 172.21.15.97:0/4068633000 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "17236019-5f1b-4e4f-b3ab-bb0b664551b1", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f96a8006720 con 0x7f96ac0b9160 2024-03-14T21:13:40.714 INFO:teuthology.orchestra.run.smithi097.stdout:6 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 msgr2=0x7f96ac1e1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac1e1650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f96a8010d10 tx=0x7f96a800b510 comp rx=0 tx=0).stop 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 shutdown_connections 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96ac145e00 0x7f96ac0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96ac0b9160 0x7f96ac1e1650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 --2- 172.21.15.97:0/4068633000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96ac0b8d20 0x7f96ac0be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 >> 172.21.15.97:0/4068633000 conn(0x7f96ac06a310 msgr2=0x7f96ac146360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 shutdown_connections 2024-03-14T21:13:40.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.732+0000 7f96b3c84640 1 -- 172.21.15.97:0/4068633000 wait complete. 2024-03-14T21:13:40.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new e9bae753-4d72-434b-8919-02054a33b4d2 7 2024-03-14T21:13:40.780 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.797+0000 7feb37a1e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-14T21:13:40.787 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.804+0000 7f68d8e21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-14T21:13:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.844+0000 7fdf4a53e640 1 Processor -- start 2024-03-14T21:13:40.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.844+0000 7fdf4a53e640 1 -- start start 2024-03-14T21:13:40.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf4a53e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdf440b7e00 0x7fdf440b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf4a53e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf4406a750 0x7fdf441476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf4a53e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4406ab90 con 0x7fdf44147bf0 2024-03-14T21:13:40.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf4a53e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf440b2d40 con 0x7fdf440b7e00 2024-03-14T21:13:40.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf4a53e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440b1770 con 0x7fdf4406a750 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf4406a750 0x7fdf441476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf4406a750 0x7fdf441476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:46960/0 (socket says 172.21.15.97:46960) 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf437fe640 1 -- 172.21.15.97:0/246879947 learned_addr learned my addr 172.21.15.97:0/246879947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.845+0000 7fdf43fff640 1 --2- 172.21.15.97:0/246879947 >> v2:172.21.15.49:3300/0 conn(0x7fdf440b7e00 0x7fdf440b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.846+0000 7fdf437fe640 1 -- 172.21.15.97:0/246879947 >> v2:172.21.15.49:3300/0 conn(0x7fdf440b7e00 msgr2=0x7fdf440b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.846+0000 7fdf437fe640 1 --2- 172.21.15.97:0/246879947 >> v2:172.21.15.49:3300/0 conn(0x7fdf440b7e00 0x7fdf440b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.846+0000 7fdf437fe640 1 -- 172.21.15.97:0/246879947 >> v1:172.21.15.49:6789/0 conn(0x7fdf44147bf0 legacy=0x7fdf44141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.846+0000 7fdf437fe640 1 -- 172.21.15.97:0/246879947 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440b2540 con 0x7fdf4406a750 2024-03-14T21:13:40.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.846+0000 7fdf437fe640 1 --2- 172.21.15.97:0/246879947 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf4406a750 0x7fdf441476b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46eaf6633c0130f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.847+0000 7fdf42ffd640 1 -- 172.21.15.97:0/246879947 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3402f840 con 0x7fdf4406a750 2024-03-14T21:13:40.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.847+0000 7fdf42ffd640 1 -- 172.21.15.97:0/246879947 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3402f9a0 con 0x7fdf4406a750 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.847+0000 7fdf42ffd640 1 -- 172.21.15.97:0/246879947 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3402fd30 con 0x7fdf4406a750 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.847+0000 7fdf4a53e640 1 -- 172.21.15.97:0/246879947 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf4406a750 msgr2=0x7fdf441476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.847+0000 7fdf4a53e640 1 --2- 172.21.15.97:0/246879947 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf4406a750 0x7fdf441476b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3402ba20 comp rx=0 tx=0).stop 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.848+0000 7fdf4a53e640 1 -- 172.21.15.97:0/246879947 shutdown_connections 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.848+0000 7fdf4a53e640 1 --2- 172.21.15.97:0/246879947 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf4406a750 0x7fdf441476b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.848+0000 7fdf4a53e640 1 --2- 172.21.15.97:0/246879947 >> v2:172.21.15.49:3300/0 conn(0x7fdf440b7e00 0x7fdf440b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.848+0000 7fdf4a53e640 1 -- 172.21.15.97:0/246879947 >> 172.21.15.97:0/246879947 conn(0x7fdf440bca30 msgr2=0x7fdf44069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.848+0000 7fdf4a53e640 1 -- 172.21.15.97:0/246879947 shutdown_connections 2024-03-14T21:13:40.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.848+0000 7fdf4a53e640 1 -- 172.21.15.97:0/246879947 wait complete. 2024-03-14T21:13:40.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.849+0000 7fdf4a53e640 1 Processor -- start 2024-03-14T21:13:40.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.849+0000 7fdf4a53e640 1 -- start start 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.850+0000 7fdf4a53e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4406a750 0x7fdf441d3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.850+0000 7fdf4a53e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf440b7e00 0x7fdf44071860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.850+0000 7fdf4a53e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf44147bf0 0x7fdf441dbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.850+0000 7fdf4a53e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf441d3540 con 0x7fdf44147bf0 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.850+0000 7fdf4a53e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf44070b50 con 0x7fdf4406a750 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.850+0000 7fdf4a53e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4406f610 con 0x7fdf440b7e00 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf43fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4406a750 0x7fdf441d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf440b7e00 0x7fdf44071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf43fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4406a750 0x7fdf441d3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34244/0 (socket says 172.21.15.97:34244) 2024-03-14T21:13:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf43fff640 1 -- 172.21.15.97:0/779098036 learned_addr learned my addr 172.21.15.97:0/779098036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf48adb640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf44147bf0 0x7fdf441dbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf437fe640 1 -- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4406a750 msgr2=0x7fdf441d3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf437fe640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4406a750 0x7fdf441d3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf437fe640 1 -- 172.21.15.97:0/779098036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf44147bf0 msgr2=0x7fdf441dbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf437fe640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf44147bf0 0x7fdf441dbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf437fe640 1 -- 172.21.15.97:0/779098036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf340094d0 con 0x7fdf440b7e00 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf43fff640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4406a750 0x7fdf441d3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf437fe640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf440b7e00 0x7fdf44071860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf440b18b0 tx=0x7fdf34004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf40ff9640 1 -- 172.21.15.97:0/779098036 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34037bc0 con 0x7fdf440b7e00 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf40ff9640 1 -- 172.21.15.97:0/779098036 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf34037d20 con 0x7fdf440b7e00 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4406f3c0 con 0x7fdf440b7e00 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf40ff9640 1 -- 172.21.15.97:0/779098036 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf340426c0 con 0x7fdf440b7e00 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf440b36d0 con 0x7fdf440b7e00 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf40ff9640 1 -- 172.21.15.97:0/779098036 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdf34042b20 con 0x7fdf440b7e00 2024-03-14T21:13:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.851+0000 7fdf40ff9640 1 -- 172.21.15.97:0/779098036 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fdf3403c070 con 0x7fdf440b7e00 2024-03-14T21:13:40.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.852+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf44070d00 con 0x7fdf440b7e00 2024-03-14T21:13:40.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.854+0000 7fdf40ff9640 1 -- 172.21.15.97:0/779098036 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fdf44070d00 con 0x7fdf440b7e00 2024-03-14T21:13:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.924+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e9bae753-4d72-434b-8919-02054a33b4d2", "id": 7} v 0) v1 -- 0x7fdf44071da0 con 0x7fdf440b7e00 2024-03-14T21:13:40.943 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:40.938+0000 7feb37a1e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-14T21:13:40.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.927+0000 7fdf40ff9640 1 -- 172.21.15.97:0/779098036 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9bae753-4d72-434b-8919-02054a33b4d2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdf34047070 con 0x7fdf440b7e00 2024-03-14T21:13:40.943 INFO:teuthology.orchestra.run.smithi097.stdout:7 2024-03-14T21:13:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf440b7e00 msgr2=0x7fdf44071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf440b7e00 0x7fdf44071860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf440b18b0 tx=0x7fdf34004730 comp rx=0 tx=0).stop 2024-03-14T21:13:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 shutdown_connections 2024-03-14T21:13:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf44147bf0 0x7fdf441dbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf440b7e00 0x7fdf44071860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 --2- 172.21.15.97:0/779098036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4406a750 0x7fdf441d3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:40.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 >> 172.21.15.97:0/779098036 conn(0x7fdf440bca30 msgr2=0x7fdf4406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:40.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 shutdown_connections 2024-03-14T21:13:40.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-14T21:13:40.928+0000 7fdf4a53e640 1 -- 172.21.15.97:0/779098036 wait complete. 2024-03-14T21:13:40.946 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-14T21:13:40.946 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-14T21:13:40.946 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:40.946+0000 7f68d8e21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-14T21:13:40.948 INFO:tasks.ceph.osd.0:Started 2024-03-14T21:13:40.948 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-14T21:13:40.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-14T21:13:40.951 INFO:tasks.ceph.osd.1:Started 2024-03-14T21:13:40.951 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-14T21:13:40.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-14T21:13:40.953 INFO:tasks.ceph.osd.2:Started 2024-03-14T21:13:40.953 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-14T21:13:40.953 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-14T21:13:40.955 INFO:tasks.ceph.osd.3:Started 2024-03-14T21:13:40.955 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-14T21:13:40.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-14T21:13:40.958 INFO:tasks.ceph.osd.4:Started 2024-03-14T21:13:40.958 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-14T21:13:40.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-14T21:13:40.960 INFO:tasks.ceph.osd.5:Started 2024-03-14T21:13:40.960 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-14T21:13:40.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-14T21:13:40.962 INFO:tasks.ceph.osd.6:Started 2024-03-14T21:13:40.962 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-14T21:13:40.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-14T21:13:40.966 INFO:tasks.ceph.osd.7:Started 2024-03-14T21:13:40.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:13:41.070 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:41.086+0000 7f68d8e21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-14T21:13:41.072 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:41.089+0000 7feb37a1e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-14T21:13:41.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.111+0000 7fb1296e9640 1 Processor -- start 2024-03-14T21:13:41.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.111+0000 7fb1296e9640 1 -- start start 2024-03-14T21:13:41.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb1296e9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb12406adf0 0x7fb12414ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:41.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb1296e9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406b1d0 0x7fb12414b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:41.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb1296e9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1241451a0 con 0x7fb12406c230 2024-03-14T21:13:41.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb1296e9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12406e990 con 0x7fb12406adf0 2024-03-14T21:13:41.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb1296e9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12406d960 con 0x7fb12406b1d0 2024-03-14T21:13:41.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb122ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb12406adf0 0x7fb12414ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb122ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb12406adf0 0x7fb12414ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59824/0 (socket says 172.21.15.49:59824) 2024-03-14T21:13:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb122ffd640 1 -- 172.21.15.49:0/3993196509 learned_addr learned my addr 172.21.15.49:0/3993196509 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb1227fc640 1 --1- 172.21.15.49:0/3993196509 >> v1:172.21.15.49:6789/0 conn(0x7fb12406c230 0x7fb12414df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56236/0 (socket says 172.21.15.49:56236) 2024-03-14T21:13:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb1237fe640 1 --2- 172.21.15.49:0/3993196509 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406b1d0 0x7fb12414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.112+0000 7fb122ffd640 1 -- 172.21.15.49:0/3993196509 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406b1d0 msgr2=0x7fb12414b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb122ffd640 1 --2- 172.21.15.49:0/3993196509 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406b1d0 0x7fb12414b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb122ffd640 1 -- 172.21.15.49:0/3993196509 >> v1:172.21.15.49:6789/0 conn(0x7fb12406c230 legacy=0x7fb12414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb122ffd640 1 -- 172.21.15.49:0/3993196509 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12406dc40 con 0x7fb12406adf0 2024-03-14T21:13:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb121ffb640 1 -- 172.21.15.49:0/3993196509 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249818960 0 0) 0x7fb1241451a0 con 0x7fb12406c230 2024-03-14T21:13:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb1237fe640 1 --2- 172.21.15.49:0/3993196509 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406b1d0 0x7fb12414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb122ffd640 1 --2- 172.21.15.49:0/3993196509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb12414ae50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb118009870 tx=0x7fb11802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea986795cd13347c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb121ffb640 1 -- 172.21.15.49:0/3993196509 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118030860 con 0x7fb12406adf0 2024-03-14T21:13:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb121ffb640 1 -- 172.21.15.49:0/3993196509 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1180309c0 con 0x7fb12406adf0 2024-03-14T21:13:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.113+0000 7fb121ffb640 1 -- 172.21.15.49:0/3993196509 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118030d50 con 0x7fb12406adf0 2024-03-14T21:13:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 -- 172.21.15.49:0/3993196509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 msgr2=0x7fb12414ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 --2- 172.21.15.49:0/3993196509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb12414ae50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb118009870 tx=0x7fb11802b9a0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 -- 172.21.15.49:0/3993196509 shutdown_connections 2024-03-14T21:13:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 --2- 172.21.15.49:0/3993196509 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406b1d0 0x7fb12414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 --2- 172.21.15.49:0/3993196509 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb12414ae50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 -- 172.21.15.49:0/3993196509 >> 172.21.15.49:0/3993196509 conn(0x7fb1240bad40 msgr2=0x7fb1240bb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 -- 172.21.15.49:0/3993196509 shutdown_connections 2024-03-14T21:13:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.114+0000 7fb1296e9640 1 -- 172.21.15.49:0/3993196509 wait complete. 2024-03-14T21:13:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.115+0000 7fb1296e9640 1 Processor -- start 2024-03-14T21:13:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.115+0000 7fb1296e9640 1 -- start start 2024-03-14T21:13:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1296e9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb1241e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1296e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12406b1d0 0x7fb1240b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1296e9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406c230 0x7fb1240a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb1241e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1296e9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240a34f0 con 0x7fb12406adf0 2024-03-14T21:13:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1296e9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12409d5d0 con 0x7fb12406b1d0 2024-03-14T21:13:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1296e9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12409d7e0 con 0x7fb12406c230 2024-03-14T21:13:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb1241e3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59830/0 (socket says 172.21.15.49:59830) 2024-03-14T21:13:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 -- 172.21.15.49:0/2470983757 learned_addr learned my addr 172.21.15.49:0/2470983757 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1227fc640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12406b1d0 0x7fb1240b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb1237fe640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406c230 0x7fb1240a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 -- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406c230 msgr2=0x7fb1240a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406c230 0x7fb1240a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 -- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12406b1d0 msgr2=0x7fb1240b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12406b1d0 0x7fb1240b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.116+0000 7fb122ffd640 1 -- 172.21.15.49:0/2470983757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c009a70 con 0x7fb12406adf0 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb122ffd640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb1241e3f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb118009870 tx=0x7fb118004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb10b7fe640 1 -- 172.21.15.49:0/2470983757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118039c90 con 0x7fb12406adf0 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb10b7fe640 1 -- 172.21.15.49:0/2470983757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118039e30 con 0x7fb12406adf0 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb10b7fe640 1 -- 172.21.15.49:0/2470983757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118043640 con 0x7fb12406adf0 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1180094d0 con 0x7fb12406adf0 2024-03-14T21:13:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1241e3700 con 0x7fb12406adf0 2024-03-14T21:13:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb10b7fe640 1 -- 172.21.15.49:0/2470983757 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb118043aa0 con 0x7fb12406adf0 2024-03-14T21:13:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.117+0000 7fb10b7fe640 1 -- 172.21.15.49:0/2470983757 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb11803e070 con 0x7fb12406adf0 2024-03-14T21:13:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.118+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb124073540 con 0x7fb12406adf0 2024-03-14T21:13:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.120+0000 7fb10b7fe640 1 -- 172.21.15.49:0/2470983757 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fb118034070 con 0x7fb12406adf0 2024-03-14T21:13:41.115 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:41.132+0000 7f68d8e21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-14T21:13:41.121 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:41.138+0000 7feb37a1e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-14T21:13:41.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.181+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb12406be30 con 0x7fb12406adf0 2024-03-14T21:13:41.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.181+0000 7fb10b7fe640 1 -- 172.21.15.49:0/2470983757 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb11804d460 con 0x7fb12406adf0 2024-03-14T21:13:41.165 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:41.165 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:13:40.926498+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 msgr2=0x7fb1241e3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb1241e3f00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb118009870 tx=0x7fb118004730 comp rx=0 tx=0).stop 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 shutdown_connections 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12406c230 0x7fb1240a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb12406b1d0 0x7fb1240b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 --2- 172.21.15.49:0/2470983757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb12406adf0 0x7fb1241e3f00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 >> 172.21.15.49:0/2470983757 conn(0x7fb1240bad40 msgr2=0x7fb1240b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 shutdown_connections 2024-03-14T21:13:41.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:41.182+0000 7fb1296e9640 1 -- 172.21.15.49:0/2470983757 wait complete. 2024-03-14T21:13:41.176 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-14T21:13:41.176 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-14T21:13:41.224 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:41.240+0000 7f68d8e21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-14T21:13:41.239 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:41.256+0000 7feb37a1e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-14T21:13:41.318 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:41.335+0000 7f68d8e21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-14T21:13:41.338 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:41.355+0000 7feb37a1e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-14T21:13:41.470 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:41.486+0000 7f68d8e21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-14T21:13:41.499 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:41.516+0000 7feb37a1e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-14T21:13:41.511 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:41.527+0000 7f68d8e21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-14T21:13:41.543 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:41.560+0000 7feb37a1e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-14T21:13:41.567 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-03-14T21:13:41.583+0000 7f204761f500 -1 Falling back to public interface 2024-03-14T21:13:41.567 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-03-14T21:13:41.584+0000 7f44a9425500 -1 Falling back to public interface 2024-03-14T21:13:41.568 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-03-14T21:13:41.584+0000 7f73cc214500 -1 Falling back to public interface 2024-03-14T21:13:41.580 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-03-14T21:13:41.596+0000 7f16195be500 -1 Falling back to public interface 2024-03-14T21:13:41.580 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-03-14T21:13:41.596+0000 7f6161438500 -1 Falling back to public interface 2024-03-14T21:13:41.580 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-03-14T21:13:41.597+0000 7f1618565640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-03-14T21:13:41.582 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-03-14T21:13:41.599+0000 7f11f0d9b500 -1 Falling back to public interface 2024-03-14T21:13:41.585 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-03-14T21:13:41.601+0000 7f178a9be500 -1 Falling back to public interface 2024-03-14T21:13:41.600 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-03-14T21:13:41.617+0000 7f6831616500 -1 Falling back to public interface 2024-03-14T21:13:41.665 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:41.681+0000 7f68d8e21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-14T21:13:41.708 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:41.725+0000 7feb37a1e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-14T21:13:41.981 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T21:13:42.072 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.088+0000 7fd734eca640 1 Processor -- start 2024-03-14T21:13:42.072 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.089+0000 7fd734eca640 1 -- start start 2024-03-14T21:13:42.073 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.089+0000 7fd734eca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd730142b20 0x7fd7300b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:42.073 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.089+0000 7fd734eca640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd7300b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:42.073 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.089+0000 7fd734eca640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7300b56b0 con 0x7fd730148a30 2024-03-14T21:13:42.073 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.089+0000 7fd734eca640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7300ba5f0 con 0x7fd730142b20 2024-03-14T21:13:42.073 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.089+0000 7fd734eca640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300b50d0 con 0x7fd7300b9220 2024-03-14T21:13:42.073 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72ed9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd7300b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72e59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd730142b20 0x7fd7300b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72dd9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd730148a30 0x7fd730148e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:54596/0 (socket says 172.21.15.49:54596) 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72dd9b640 1 -- 172.21.15.49:0/1782782748 learned_addr learned my addr 172.21.15.49:0/1782782748 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72e59c640 1 -- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 msgr2=0x7fd7300b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72e59c640 1 --2- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd7300b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72e59c640 1 -- 172.21.15.49:0/1782782748 >> v1:172.21.15.97:6789/0 conn(0x7fd730148a30 legacy=0x7fd730148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72e59c640 1 -- 172.21.15.49:0/1782782748 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7300b39f0 con 0x7fd730142b20 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72d59a640 1 -- 172.21.15.49:0/1782782748 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868311500 0 0) 0x7fd7300b56b0 con 0x7fd730148a30 2024-03-14T21:13:42.074 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.090+0000 7fd72ed9d640 1 --2- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd7300b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:42.075 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.091+0000 7fd72e59c640 1 --2- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730142b20 0x7fd7300b34b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097a0 tx=0x7fd72002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d65092d72e7b1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:42.075 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.091+0000 7fd72d59a640 1 -- 172.21.15.49:0/1782782748 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720030610 con 0x7fd730142b20 2024-03-14T21:13:42.075 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.091+0000 7fd72d59a640 1 -- 172.21.15.49:0/1782782748 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd720030770 con 0x7fd730142b20 2024-03-14T21:13:42.075 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.091+0000 7fd72d59a640 1 -- 172.21.15.49:0/1782782748 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720030b00 con 0x7fd730142b20 2024-03-14T21:13:42.075 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 -- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730142b20 msgr2=0x7fd7300b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:42.076 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 --2- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730142b20 0x7fd7300b34b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097a0 tx=0x7fd72002b960 comp rx=0 tx=0).stop 2024-03-14T21:13:42.076 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 -- 172.21.15.49:0/1782782748 shutdown_connections 2024-03-14T21:13:42.076 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 --2- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd7300b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.076 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 --2- 172.21.15.49:0/1782782748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730142b20 0x7fd7300b34b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.076 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 -- 172.21.15.49:0/1782782748 >> 172.21.15.49:0/1782782748 conn(0x7fd7300baf90 msgr2=0x7fd7300bb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:42.076 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 -- 172.21.15.49:0/1782782748 shutdown_connections 2024-03-14T21:13:42.076 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.092+0000 7fd734eca640 1 -- 172.21.15.49:0/1782782748 wait complete. 2024-03-14T21:13:42.078 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.093+0000 7fd734eca640 1 Processor -- start 2024-03-14T21:13:42.078 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.093+0000 7fd734eca640 1 -- start start 2024-03-14T21:13:42.078 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd73014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:42.078 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd730142b20 0x7fd7300a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:42.078 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730148a30 0x7fd730140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73014aa90 con 0x7fd730142b20 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73014bc70 con 0x7fd730148a30 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd730149d30 con 0x7fd7300b9220 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd730142b20 0x7fd7300a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd730142b20 0x7fd7300a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59838/0 (socket says 172.21.15.49:59838) 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 -- 172.21.15.49:0/2566406351 learned_addr learned my addr 172.21.15.49:0/2566406351 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72e59c640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd73014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72ed9d640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730148a30 0x7fd730140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 -- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 msgr2=0x7fd73014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd73014a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 -- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730148a30 msgr2=0x7fd730140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730148a30 0x7fd730140b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 -- 172.21.15.49:0/2566406351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728009a70 con 0x7fd730142b20 2024-03-14T21:13:42.079 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd72dd9b640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd730142b20 0x7fd7300a02a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c430 tx=0x7fd72400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd71effd640 1 -- 172.21.15.49:0/2566406351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7240074e0 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd71effd640 1 -- 172.21.15.49:0/2566406351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd724007660 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd720009400 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd71effd640 1 -- 172.21.15.49:0/2566406351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724004a20 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.094+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73014c470 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.095+0000 7fd71effd640 1 -- 172.21.15.49:0/2566406351 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd72401d3f0 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.095+0000 7fd71effd640 1 -- 172.21.15.49:0/2566406351 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd724013a90 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.095+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73014c660 con 0x7fd730142b20 2024-03-14T21:13:42.080 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.097+0000 7fd71effd640 1 -- 172.21.15.49:0/2566406351 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fd724013d30 con 0x7fd730142b20 2024-03-14T21:13:42.143 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.159+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7300672c0 con 0x7fd730142b20 2024-03-14T21:13:42.143 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.159+0000 7fd71effd640 1 -- 172.21.15.49:0/2566406351 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd7240133c0 con 0x7fd730142b20 2024-03-14T21:13:42.143 INFO:teuthology.misc.health.smithi049.stdout: 2024-03-14T21:13:42.144 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":10,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:13:40.926498+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:13:42.145 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd730142b20 msgr2=0x7fd7300a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:42.145 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd730142b20 0x7fd7300a02a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c430 tx=0x7fd72400c830 comp rx=0 tx=0).stop 2024-03-14T21:13:42.145 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 shutdown_connections 2024-03-14T21:13:42.145 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd730148a30 0x7fd730140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.146 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd730142b20 0x7fd7300a02a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.146 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 --2- 172.21.15.49:0/2566406351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd7300b9220 0x7fd73014a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:42.146 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 >> 172.21.15.49:0/2566406351 conn(0x7fd7300baf90 msgr2=0x7fd7301480b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:42.146 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 shutdown_connections 2024-03-14T21:13:42.146 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:42.160+0000 7fd734eca640 1 -- 172.21.15.49:0/2566406351 wait complete. 2024-03-14T21:13:42.153 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-14T21:13:42.435 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:42.452+0000 7f68d8e21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-14T21:13:42.552 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:42.569+0000 7feb37a1e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-14T21:13:42.705 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:42.720+0000 7f68d8e21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-14T21:13:42.754 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:42.770+0000 7f68d8e21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-14T21:13:42.837 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:42.854+0000 7feb37a1e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-14T21:13:42.868 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:42.884+0000 7f68d8e21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-14T21:13:42.892 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:42.909+0000 7feb37a1e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-14T21:13:43.015 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-03-14T21:13:43.032+0000 7feb37a1e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-14T21:13:43.897 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:43.913+0000 7f687995d640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-14T21:13:43.897 INFO:tasks.ceph.mgr.x.smithi097.stderr:2024-03-14T21:13:43.913+0000 7f687995d640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-14T21:13:45.012 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-03-14T21:13:45.028+0000 7f73cc214500 -1 osd.7 0 log_to_monitors true 2024-03-14T21:13:45.027 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-03-14T21:13:45.043+0000 7f44a9425500 -1 osd.1 0 log_to_monitors true 2024-03-14T21:13:45.057 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-03-14T21:13:45.073+0000 7f204761f500 -1 osd.6 0 log_to_monitors true 2024-03-14T21:13:45.114 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-03-14T21:13:45.130+0000 7f16195be500 -1 osd.4 0 log_to_monitors true 2024-03-14T21:13:45.114 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-03-14T21:13:45.130+0000 7f6161438500 -1 osd.5 0 log_to_monitors true 2024-03-14T21:13:45.114 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-03-14T21:13:45.130+0000 7f178a9be500 -1 osd.0 0 log_to_monitors true 2024-03-14T21:13:45.115 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-03-14T21:13:45.130+0000 7f6831616500 -1 osd.2 0 log_to_monitors true 2024-03-14T21:13:45.120 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-03-14T21:13:45.136+0000 7f11f0d9b500 -1 osd.3 0 log_to_monitors true 2024-03-14T21:13:48.550 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-03-14T21:13:48.566+0000 7f44a4b8f640 -1 osd.1 0 waiting for initial osdmap 2024-03-14T21:13:48.565 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-03-14T21:13:48.581+0000 7f449d1c2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:48.573 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-03-14T21:13:48.589+0000 7f11ec503640 -1 osd.3 0 waiting for initial osdmap 2024-03-14T21:13:48.583 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-03-14T21:13:48.599+0000 7f11e4b36640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:48.585 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-03-14T21:13:48.601+0000 7f1786126640 -1 osd.0 0 waiting for initial osdmap 2024-03-14T21:13:48.601 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-03-14T21:13:48.617+0000 7f682e01a640 -1 osd.2 0 waiting for initial osdmap 2024-03-14T21:13:48.602 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-03-14T21:13:48.618+0000 7f177e759640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:48.615 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-03-14T21:13:48.631+0000 7f6824e12640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:48.961 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T21:13:48.967 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-03-14T21:13:48.984+0000 7f615cba2640 -1 osd.5 0 waiting for initial osdmap 2024-03-14T21:13:48.977 INFO:tasks.ceph.osd.5.smithi097.stderr:2024-03-14T21:13:48.994+0000 7f61551d5640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:48.986 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-03-14T21:13:49.002+0000 7f20426c7640 -1 osd.6 0 waiting for initial osdmap 2024-03-14T21:13:49.004 INFO:tasks.ceph.osd.6.smithi097.stderr:2024-03-14T21:13:49.021+0000 7f203acfa640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:49.011 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-03-14T21:13:49.028+0000 7f73c72b9640 -1 osd.7 0 waiting for initial osdmap 2024-03-14T21:13:49.026 INFO:tasks.ceph.osd.7.smithi097.stderr:2024-03-14T21:13:49.043+0000 7f73bf8ec640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:49.049 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-03-14T21:13:49.065+0000 7f1614d26640 -1 osd.4 0 waiting for initial osdmap 2024-03-14T21:13:49.056 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.072+0000 7ff446355640 1 Processor -- start 2024-03-14T21:13:49.056 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.072+0000 7ff446355640 1 -- start start 2024-03-14T21:13:49.063 INFO:tasks.ceph.osd.4.smithi097.stderr:2024-03-14T21:13:49.076+0000 7f160d359640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.072+0000 7ff446355640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4400bc810 0x7ff44006b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.072+0000 7ff446355640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4400bc3f0 con 0x7ff4400bb0e0 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.072+0000 7ff446355640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44014f120 con 0x7ff4400bf740 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.072+0000 7ff446355640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44014bd90 con 0x7ff4400bc810 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff4448f2640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff4400bb0e0 0x7ff44014ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:56244/0 (socket says 172.21.15.49:56244) 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff4448f2640 1 -- 172.21.15.49:0/892964182 learned_addr learned my addr 172.21.15.49:0/892964182 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43f7fe640 1 --2- 172.21.15.49:0/892964182 >> v2:172.21.15.97:3300/0 conn(0x7ff4400bc810 0x7ff44006b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234292598 0 0) 0x7ff4400bc3f0 con 0x7ff4400bb0e0 2024-03-14T21:13:49.064 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff428003520 con 0x7ff4400bb0e0 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4050615644 0 0) 0x7ff428003520 con 0x7ff4400bb0e0 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 >> v2:172.21.15.97:3300/0 conn(0x7ff4400bc810 msgr2=0x7ff44006b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 --2- 172.21.15.49:0/892964182 >> v2:172.21.15.97:3300/0 conn(0x7ff4400bc810 0x7ff44006b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 >> v1:172.21.15.97:6789/0 conn(0x7ff4400bf740 legacy=0x7ff44014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400bfe60 con 0x7ff4400bb0e0 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7ff4380019f0 con 0x7ff4400bb0e0 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 >> v1:172.21.15.49:6789/0 conn(0x7ff4400bb0e0 legacy=0x7ff44014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff428003970 0x7ff428003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4280047c0 0x7ff428004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff428005180 0x7ff42800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44014bd90 con 0x7ff428005180 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42800c020 con 0x7ff4280047c0 2024-03-14T21:13:49.065 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42800c1e0 con 0x7ff428003970 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff4448f2640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff428005180 0x7ff42800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43f7fe640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4280047c0 0x7ff428004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.073+0000 7ff43ffff640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff428003970 0x7ff428003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff4448f2640 1 -- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff428003970 msgr2=0x7ff428003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff4448f2640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff428003970 0x7ff428003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff4448f2640 1 -- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4280047c0 msgr2=0x7ff428004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff4448f2640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4280047c0 0x7ff428004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff4448f2640 1 -- 172.21.15.49:0/892964182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff428003520 con 0x7ff428005180 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff43f7fe640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4280047c0 0x7ff428004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff4448f2640 1 -- 172.21.15.49:0/892964182 reap_dead start 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff4448f2640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff428005180 0x7ff42800b980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff43800a830 tx=0x7ff4280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d923d2631906e8d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380072d0 con 0x7ff428005180 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43802f070 con 0x7ff428005180 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.074+0000 7ff43effd640 1 -- 172.21.15.49:0/892964182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380076b0 con 0x7ff428005180 2024-03-14T21:13:49.066 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.075+0000 7ff446355640 1 -- 172.21.15.49:0/892964182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff428005180 msgr2=0x7ff42800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.075+0000 7ff446355640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff428005180 0x7ff42800b980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff43800a830 tx=0x7ff4280057f0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.075+0000 7ff446355640 1 -- 172.21.15.49:0/892964182 shutdown_connections 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.075+0000 7ff446355640 1 --2- 172.21.15.49:0/892964182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff428005180 0x7ff42800b980 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.075+0000 7ff446355640 1 -- 172.21.15.49:0/892964182 >> 172.21.15.49:0/892964182 conn(0x7ff4400c0290 msgr2=0x7ff4400c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.075+0000 7ff446355640 1 -- 172.21.15.49:0/892964182 shutdown_connections 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.075+0000 7ff446355640 1 -- 172.21.15.49:0/892964182 wait complete. 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.076+0000 7ff446355640 1 Processor -- start 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.076+0000 7ff446355640 1 -- start start 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff446355640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff428005180 0x7ff4400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff446355640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff44014df20 0x7ff440144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.067 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff446355640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff440144f60 0x7ff4401e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff446355640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4400a42b0 con 0x7ff440144f60 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff446355640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4400a5640 con 0x7ff428005180 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff446355640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4401e3c40 con 0x7ff44014df20 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff440144f60 0x7ff4401e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff440144f60 0x7ff4401e2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59898/0 (socket says 172.21.15.49:59898) 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 -- 172.21.15.49:0/1392184436 learned_addr learned my addr 172.21.15.49:0/1392184436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff43f7fe640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff44014df20 0x7ff440144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff43ffff640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff428005180 0x7ff4400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 -- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff44014df20 msgr2=0x7ff440144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff44014df20 0x7ff440144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 -- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff428005180 msgr2=0x7ff4400a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:49.068 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff428005180 0x7ff4400a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 -- 172.21.15.49:0/1392184436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400a44c0 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff43f7fe640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff44014df20 0x7ff440144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff43ffff640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff428005180 0x7ff4400a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.077+0000 7ff4448f2640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff440144f60 0x7ff4401e2f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff438000f80 tx=0x7ff438001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.078+0000 7ff43cff9640 1 -- 172.21.15.49:0/1392184436 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380019b0 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.078+0000 7ff43cff9640 1 -- 172.21.15.49:0/1392184436 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438004b60 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.078+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4400a47a0 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.078+0000 7ff43cff9640 1 -- 172.21.15.49:0/1392184436 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438007400 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.078+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4400a53e0 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.079+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff440066c20 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.079+0000 7ff43cff9640 1 -- 172.21.15.49:0/1392184436 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 88763+0+0 (secure 0 0 0) 0x7ff438031030 con 0x7ff440144f60 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.080+0000 7ff43cff9640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff42c06a740 0x7ff42c06cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.080+0000 7ff43ffff640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff42c06a740 0x7ff42c06cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:49.069 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.080+0000 7ff43cff9640 1 -- 172.21.15.49:0/1392184436 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7ff4380a9890 con 0x7ff440144f60 2024-03-14T21:13:49.070 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.080+0000 7ff43ffff640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff42c06a740 0x7ff42c06cc00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff434000c00 tx=0x7ff434009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:49.070 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.084+0000 7ff43cff9640 1 -- 172.21.15.49:0/1392184436 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff438075bb0 con 0x7ff440144f60 2024-03-14T21:13:49.249 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.265+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff440066c20 con 0x7ff440144f60 2024-03-14T21:13:49.250 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.266+0000 7ff43cff9640 1 -- 172.21.15.49:0/1392184436 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff4380791d0 con 0x7ff440144f60 2024-03-14T21:13:49.250 INFO:teuthology.misc.health.smithi049.stdout: 2024-03-14T21:13:49.250 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":12,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:13:46.917425+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:13:49.252 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.267+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff42c06a740 msgr2=0x7ff42c06cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:49.252 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.267+0000 7ff446355640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff42c06a740 0x7ff42c06cc00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff434000c00 tx=0x7ff434009040 comp rx=0 tx=0).stop 2024-03-14T21:13:49.252 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.267+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff440144f60 msgr2=0x7ff4401e2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.267+0000 7ff446355640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff440144f60 0x7ff4401e2f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff438000f80 tx=0x7ff438001d80 comp rx=0 tx=0).stop 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 shutdown_connections 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff42c06a740 0x7ff42c06cc00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff440144f60 0x7ff4401e2f00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff44014df20 0x7ff440144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 --2- 172.21.15.49:0/1392184436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff428005180 0x7ff4400a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 >> 172.21.15.49:0/1392184436 conn(0x7ff4400c0290 msgr2=0x7ff44014e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 shutdown_connections 2024-03-14T21:13:49.253 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:49.268+0000 7ff446355640 1 -- 172.21.15.49:0/1392184436 wait complete. 2024-03-14T21:13:49.261 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-14T21:13:56.069 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T21:13:56.172 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.188+0000 7f899c08b640 1 Processor -- start 2024-03-14T21:13:56.172 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.188+0000 7f899c08b640 1 -- start start 2024-03-14T21:13:56.173 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f899c08b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bc810 0x7f899406b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.173 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f899c08b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f89940bb0e0 0x7f899414ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.173 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f899c08b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89940bc3f0 con 0x7f89940bf740 2024-03-14T21:13:56.173 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f899c08b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899414bd90 con 0x7f89940bb0e0 2024-03-14T21:13:56.173 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f899c08b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89940bfe60 con 0x7f89940bc810 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f899a628640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f89940bb0e0 0x7f899414ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f8999626640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bc810 0x7f899406b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f8999e27640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f89940bf740 0x7f899414e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:46406/0 (socket says 172.21.15.49:46406) 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.189+0000 7f8999e27640 1 -- 172.21.15.49:0/138086013 learned_addr learned my addr 172.21.15.49:0/138086013 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f8998e25640 1 -- 172.21.15.49:0/138086013 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055245625 0 0) 0x7f89940bc3f0 con 0x7f89940bf740 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f8998e25640 1 -- 172.21.15.49:0/138086013 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8980003520 con 0x7f89940bf740 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f8999626640 1 -- 172.21.15.49:0/138086013 >> v2:172.21.15.97:3300/0 conn(0x7f89940bb0e0 msgr2=0x7f899414ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f8999626640 1 --2- 172.21.15.49:0/138086013 >> v2:172.21.15.97:3300/0 conn(0x7f89940bb0e0 0x7f899414ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f8999626640 1 -- 172.21.15.49:0/138086013 >> v1:172.21.15.97:6789/0 conn(0x7f89940bf740 legacy=0x7f899414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f8999626640 1 -- 172.21.15.49:0/138086013 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899406aad0 con 0x7f89940bc810 2024-03-14T21:13:56.175 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f899a628640 1 --2- 172.21.15.49:0/138086013 >> v2:172.21.15.97:3300/0 conn(0x7f89940bb0e0 0x7f899414ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:56.176 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.190+0000 7f8999626640 1 --2- 172.21.15.49:0/138086013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bc810 0x7f899406b2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f89840097a0 tx=0x7f898402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1e97c1488354b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.176 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f8998e25640 1 -- 172.21.15.49:0/138086013 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984031830 con 0x7f89940bc810 2024-03-14T21:13:56.176 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f8998e25640 1 -- 172.21.15.49:0/138086013 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8984031990 con 0x7f89940bc810 2024-03-14T21:13:56.176 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f8998e25640 1 -- 172.21.15.49:0/138086013 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984031d20 con 0x7f89940bc810 2024-03-14T21:13:56.178 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 -- 172.21.15.49:0/138086013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bc810 msgr2=0x7f899406b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.178 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 --2- 172.21.15.49:0/138086013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bc810 0x7f899406b2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f89840097a0 tx=0x7f898402b8d0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.178 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 -- 172.21.15.49:0/138086013 shutdown_connections 2024-03-14T21:13:56.178 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 --2- 172.21.15.49:0/138086013 >> v2:172.21.15.97:3300/0 conn(0x7f89940bb0e0 0x7f899414ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.178 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 --2- 172.21.15.49:0/138086013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bc810 0x7f899406b2d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.178 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 -- 172.21.15.49:0/138086013 >> 172.21.15.49:0/138086013 conn(0x7f89940c0290 msgr2=0x7f89940c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:56.178 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 -- 172.21.15.49:0/138086013 shutdown_connections 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.191+0000 7f899c08b640 1 -- 172.21.15.49:0/138086013 wait complete. 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.192+0000 7f899c08b640 1 Processor -- start 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.192+0000 7f899c08b640 1 -- start start 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f899c08b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940bb0e0 0x7f89940a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f899c08b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89940bc810 0x7f899414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f899c08b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bf740 0x7f89941dfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f899c08b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89940a5fd0 con 0x7f89940bc810 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f899c08b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899414c2d0 con 0x7f89940bb0e0 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f899c08b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899414c5b0 con 0x7f89940bf740 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999626640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89940bc810 0x7f899414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999626640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89940bc810 0x7f899414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36558/0 (socket says 172.21.15.49:36558) 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999626640 1 -- 172.21.15.49:0/3361812512 learned_addr learned my addr 172.21.15.49:0/3361812512 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f899a628640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bf740 0x7f89941dfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999e27640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940bb0e0 0x7f89940a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.179 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999626640 1 -- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bf740 msgr2=0x7f89941dfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999626640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bf740 0x7f89941dfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999626640 1 -- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940bb0e0 msgr2=0x7f89940a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.193+0000 7f8999626640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940bb0e0 0x7f89940a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.194+0000 7f8999626640 1 -- 172.21.15.49:0/3361812512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c009a70 con 0x7f89940bc810 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.194+0000 7f8999626640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89940bc810 0x7f899414d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f89840098c0 tx=0x7f89840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.194+0000 7f898a7fc640 1 -- 172.21.15.49:0/3361812512 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898403a630 con 0x7f89940bc810 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.194+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8984009400 con 0x7f89940bc810 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.194+0000 7f898a7fc640 1 -- 172.21.15.49:0/3361812512 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898403a7b0 con 0x7f89940bc810 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.194+0000 7f898a7fc640 1 -- 172.21.15.49:0/3361812512 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898403ab60 con 0x7f89940bc810 2024-03-14T21:13:56.180 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.194+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89940bc140 con 0x7f89940bc810 2024-03-14T21:13:56.181 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.195+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899414d810 con 0x7f89940bc810 2024-03-14T21:13:56.181 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.195+0000 7f898a7fc640 1 -- 172.21.15.49:0/3361812512 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f898403acc0 con 0x7f89940bc810 2024-03-14T21:13:56.181 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.196+0000 7f898a7fc640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f897806a7a0 0x7f897806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.181 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.196+0000 7f898a7fc640 1 -- 172.21.15.49:0/3361812512 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f89840aa2e0 con 0x7f89940bc810 2024-03-14T21:13:56.181 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.196+0000 7f8999e27640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f897806a7a0 0x7f897806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.181 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.196+0000 7f8999e27640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f897806a7a0 0x7f897806cc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8990005880 tx=0x7f8990009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.184 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.200+0000 7f898a7fc640 1 -- 172.21.15.49:0/3361812512 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f89840759f0 con 0x7f89940bc810 2024-03-14T21:13:56.366 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.382+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8994066c20 con 0x7f89940bc810 2024-03-14T21:13:56.367 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.383+0000 7f898a7fc640 1 -- 172.21.15.49:0/3361812512 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f8984079010 con 0x7f89940bc810 2024-03-14T21:13:56.367 INFO:teuthology.misc.health.smithi049.stdout: 2024-03-14T21:13:56.368 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":17,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:13:53.613681+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f897806a7a0 msgr2=0x7f897806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f897806a7a0 0x7f897806cc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8990005880 tx=0x7f8990009040 comp rx=0 tx=0).stop 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89940bc810 msgr2=0x7f899414d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89940bc810 0x7f899414d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f89840098c0 tx=0x7f89840046c0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 shutdown_connections 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f897806a7a0 0x7f897806cc60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89940bf740 0x7f89941dfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89940bc810 0x7f899414d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 --2- 172.21.15.49:0/3361812512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89940bb0e0 0x7f89940a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 >> 172.21.15.49:0/3361812512 conn(0x7f89940c0290 msgr2=0x7f8994149cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 shutdown_connections 2024-03-14T21:13:56.370 INFO:teuthology.misc.health.smithi049.stderr:2024-03-14T21:13:56.385+0000 7f899c08b640 1 -- 172.21.15.49:0/3361812512 wait complete. 2024-03-14T21:13:56.379 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-14T21:13:56.379 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-14T21:13:56.379 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T21:13:56.379 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T21:13:56.381 INFO:tasks.ceph.mds.a:Started 2024-03-14T21:13:56.382 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T21:13:56.382 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T21:13:56.384 INFO:tasks.ceph.mds.c:Started 2024-03-14T21:13:56.384 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T21:13:56.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T21:13:56.387 INFO:tasks.ceph.mds.b:Started 2024-03-14T21:13:56.387 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-14T21:13:56.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:13:56.464 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2024-03-14T21:13:56.465 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-03-14T21:13:56.470 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2024-03-14T21:13:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe455838640 1 Processor -- start 2024-03-14T21:13:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe455838640 1 -- start start 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe455838640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 0x7fe45006a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe455838640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe45006ae50 0x7fe450155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe455838640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4500bb310 con 0x7fe45014b710 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe455838640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4500bbab0 con 0x7fe45006ae50 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe455838640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45014aae0 con 0x7fe4500bc8c0 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe44f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe45006ae50 0x7fe450155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe44effd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe45014b710 0x7fe4500bac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55618/0 (socket says 172.21.15.49:55618) 2024-03-14T21:13:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.513+0000 7fe44effd640 1 -- 172.21.15.49:0/1192873535 learned_addr learned my addr 172.21.15.49:0/1192873535 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:13:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.514+0000 7fe44f7fe640 1 -- 172.21.15.49:0/1192873535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 msgr2=0x7fe45006a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.514+0000 7fe44f7fe640 1 --2- 172.21.15.49:0/1192873535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 0x7fe45006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.514+0000 7fe44f7fe640 1 -- 172.21.15.49:0/1192873535 >> v1:172.21.15.49:6789/0 conn(0x7fe45014b710 legacy=0x7fe4500bac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.514+0000 7fe44f7fe640 1 -- 172.21.15.49:0/1192873535 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500bfa80 con 0x7fe45006ae50 2024-03-14T21:13:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.514+0000 7fe44dffb640 1 -- 172.21.15.49:0/1192873535 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841290326 0 0) 0x7fe4500bb310 con 0x7fe45014b710 2024-03-14T21:13:56.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.514+0000 7fe44f7fe640 1 --2- 172.21.15.49:0/1192873535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe450155310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe448009d90 tx=0x7fe44802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446f8fa2dec0f944 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe44dffb640 1 -- 172.21.15.49:0/1192873535 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44802cc80 con 0x7fe45006ae50 2024-03-14T21:13:56.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe44dffb640 1 -- 172.21.15.49:0/1192873535 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44802cde0 con 0x7fe45006ae50 2024-03-14T21:13:56.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe44dffb640 1 -- 172.21.15.49:0/1192873535 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448034660 con 0x7fe45006ae50 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 -- 172.21.15.49:0/1192873535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 msgr2=0x7fe450155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 --2- 172.21.15.49:0/1192873535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe450155310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe448009d90 tx=0x7fe44802bed0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 -- 172.21.15.49:0/1192873535 shutdown_connections 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 --2- 172.21.15.49:0/1192873535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe450155310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 --2- 172.21.15.49:0/1192873535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 0x7fe45006a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 -- 172.21.15.49:0/1192873535 >> 172.21.15.49:0/1192873535 conn(0x7fe4500c0460 msgr2=0x7fe4500c0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 -- 172.21.15.49:0/1192873535 shutdown_connections 2024-03-14T21:13:56.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.515+0000 7fe455838640 1 -- 172.21.15.49:0/1192873535 wait complete. 2024-03-14T21:13:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.517+0000 7fe455838640 1 Processor -- start 2024-03-14T21:13:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.517+0000 7fe455838640 1 -- start start 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe455838640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe45014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe455838640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 0x7fe4500a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe455838640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45014b710 0x7fe4500b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe45014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe45014da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36596/0 (socket says 172.21.15.49:36596) 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 -- 172.21.15.49:0/419940490 learned_addr learned my addr 172.21.15.49:0/419940490 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4500b4ca0 con 0x7fe45006ae50 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45014cca0 con 0x7fe45014b710 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45014cf40 con 0x7fe4500bc8c0 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44e7fc640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 0x7fe4500a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44f7fe640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45014b710 0x7fe4500b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 -- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 msgr2=0x7fe4500a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 0x7fe4500a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 -- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45014b710 msgr2=0x7fe4500b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45014b710 0x7fe4500b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448009a70 con 0x7fe45006ae50 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe44effd640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe45014da20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe44000c4d0 tx=0x7fe44000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe437fff640 1 -- 172.21.15.49:0/419940490 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400074e0 con 0x7fe45006ae50 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe437fff640 1 -- 172.21.15.49:0/419940490 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440007660 con 0x7fe45006ae50 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.518+0000 7fe437fff640 1 -- 172.21.15.49:0/419940490 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440004a20 con 0x7fe45006ae50 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.519+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45014d220 con 0x7fe45006ae50 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.519+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45014bfa0 con 0x7fe45006ae50 2024-03-14T21:13:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.519+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450073540 con 0x7fe45006ae50 2024-03-14T21:13:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.520+0000 7fe437fff640 1 -- 172.21.15.49:0/419940490 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe440004b80 con 0x7fe45006ae50 2024-03-14T21:13:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.520+0000 7fe437fff640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe43c06a7d0 0x7fe43c06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.520+0000 7fe437fff640 1 -- 172.21.15.49:0/419940490 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe440087c30 con 0x7fe45006ae50 2024-03-14T21:13:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.520+0000 7fe44e7fc640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe43c06a7d0 0x7fe43c06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.521+0000 7fe44e7fc640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe43c06a7d0 0x7fe43c06cc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4500a0510 tx=0x7fe444008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.525+0000 7fe437fff640 1 -- 172.21.15.49:0/419940490 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe440053320 con 0x7fe45006ae50 2024-03-14T21:13:56.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.706+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe45009f810 con 0x7fe45006ae50 2024-03-14T21:13:56.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.706+0000 7fe437fff640 1 -- 172.21.15.49:0/419940490 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12334 (secure 0 0 0) 0x7fe440056940 con 0x7fe45006ae50 2024-03-14T21:13:56.691 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:56.691 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:13:53.613681+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:13:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe43c06a7d0 msgr2=0x7fe43c06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe43c06a7d0 0x7fe43c06cc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4500a0510 tx=0x7fe444008040 comp rx=0 tx=0).stop 2024-03-14T21:13:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 msgr2=0x7fe45014da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe45014da20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe44000c4d0 tx=0x7fe44000c990 comp rx=0 tx=0).stop 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 shutdown_connections 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe43c06a7d0 0x7fe43c06cc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe45014b710 0x7fe4500b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4500bc8c0 0x7fe4500a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 --2- 172.21.15.49:0/419940490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe45006ae50 0x7fe45014da20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 >> 172.21.15.49:0/419940490 conn(0x7fe4500c0460 msgr2=0x7fe4500a0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 shutdown_connections 2024-03-14T21:13:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.709+0000 7fe455838640 1 -- 172.21.15.49:0/419940490 wait complete. 2024-03-14T21:13:56.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T21:13:50.964908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T21:13:56.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T21:13:56.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.828+0000 7f93b14ce640 1 Processor -- start 2024-03-14T21:13:56.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.828+0000 7f93b14ce640 1 -- start start 2024-03-14T21:13:56.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93b14ce640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93b14ce640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93ac0bb670 0x7f93ac06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93b14ce640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ac0b7590 con 0x7f93ac065d80 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93b14ce640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac072080 con 0x7f93ac0bb670 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93b14ce640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac06d070 con 0x7f93ac0bb290 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93aaffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93ab7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93ac0bb670 0x7f93ac06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93aaffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:50086/0 (socket says 172.21.15.49:50086) 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.829+0000 7f93aaffd640 1 -- 172.21.15.49:0/1239097538 learned_addr learned my addr 172.21.15.49:0/1239097538 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.830+0000 7f93aaffd640 1 -- 172.21.15.49:0/1239097538 >> v2:172.21.15.97:3300/0 conn(0x7f93ac0bb670 msgr2=0x7f93ac06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.830+0000 7f93aaffd640 1 --2- 172.21.15.49:0/1239097538 >> v2:172.21.15.97:3300/0 conn(0x7f93ac0bb670 0x7f93ac06a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.830+0000 7f93aaffd640 1 -- 172.21.15.49:0/1239097538 >> v1:172.21.15.97:6789/0 conn(0x7f93ac065d80 legacy=0x7f93ac06a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.830+0000 7f93aaffd640 1 -- 172.21.15.49:0/1239097538 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0b72b0 con 0x7f93ac0bb290 2024-03-14T21:13:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.830+0000 7f93ab7fe640 1 --2- 172.21.15.49:0/1239097538 >> v2:172.21.15.97:3300/0 conn(0x7f93ac0bb670 0x7f93ac06a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93aaffd640 1 --2- 172.21.15.49:0/1239097538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac069ce0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009e50 tx=0x7f93a402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc5768e7ac20297 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93a9ffb640 1 -- 172.21.15.49:0/1239097538 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a402cb00 con 0x7f93ac0bb290 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93a9ffb640 1 -- 172.21.15.49:0/1239097538 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a402cc60 con 0x7f93ac0bb290 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93a9ffb640 1 -- 172.21.15.49:0/1239097538 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a40343f0 con 0x7f93ac0bb290 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 -- 172.21.15.49:0/1239097538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 msgr2=0x7f93ac069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1239097538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac069ce0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009e50 tx=0x7f93a402bf60 comp rx=0 tx=0).stop 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 -- 172.21.15.49:0/1239097538 shutdown_connections 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1239097538 >> v2:172.21.15.97:3300/0 conn(0x7f93ac0bb670 0x7f93ac06a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1239097538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac069ce0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 -- 172.21.15.49:0/1239097538 >> 172.21.15.49:0/1239097538 conn(0x7f93ac0b93b0 msgr2=0x7f93ac0b97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 -- 172.21.15.49:0/1239097538 shutdown_connections 2024-03-14T21:13:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.831+0000 7f93b14ce640 1 -- 172.21.15.49:0/1239097538 wait complete. 2024-03-14T21:13:56.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 Processor -- start 2024-03-14T21:13:56.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 -- start start 2024-03-14T21:13:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac065d80 0x7f93ac1cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac1cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0bb670 0x7f93ac1df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0b3740 con 0x7f93ac065d80 2024-03-14T21:13:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac1d2500 con 0x7f93ac0bb670 2024-03-14T21:13:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93b14ce640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac1d2770 con 0x7f93ac0bb290 2024-03-14T21:13:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93aaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac065d80 0x7f93ac1cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93aaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac065d80 0x7f93ac1cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36604/0 (socket says 172.21.15.49:36604) 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.833+0000 7f93aaffd640 1 -- 172.21.15.49:0/1928291068 learned_addr learned my addr 172.21.15.49:0/1928291068 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93ab7fe640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0bb670 0x7f93ac1df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93aa7fc640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac1cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93aaffd640 1 -- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 msgr2=0x7f93ac1cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93aaffd640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac1cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93aaffd640 1 -- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0bb670 msgr2=0x7f93ac1df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93aaffd640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0bb670 0x7f93ac1df8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93aaffd640 1 -- 172.21.15.49:0/1928291068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c009400 con 0x7f93ac065d80 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93aaffd640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac065d80 0x7f93ac1cca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000f80 tx=0x7f93a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93937fe640 1 -- 172.21.15.49:0/1928291068 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4009510 con 0x7f93ac065d80 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93937fe640 1 -- 172.21.15.49:0/1928291068 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a403e450 con 0x7f93ac065d80 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a4009b30 con 0x7f93ac065d80 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93937fe640 1 -- 172.21.15.49:0/1928291068 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a403e800 con 0x7f93ac065d80 2024-03-14T21:13:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.834+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac1d2d30 con 0x7f93ac065d80 2024-03-14T21:13:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.835+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac0bc320 con 0x7f93ac065d80 2024-03-14T21:13:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.835+0000 7f93937fe640 1 -- 172.21.15.49:0/1928291068 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f93a403e960 con 0x7f93ac065d80 2024-03-14T21:13:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.836+0000 7f93937fe640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f939806a7a0 0x7f939806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.836+0000 7f93937fe640 1 -- 172.21.15.49:0/1928291068 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93a40ab770 con 0x7f93ac065d80 2024-03-14T21:13:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.836+0000 7f93aa7fc640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f939806a7a0 0x7f939806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.836+0000 7f93aa7fc640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f939806a7a0 0x7f939806cc60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93ac071b00 tx=0x7f93a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:56.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:56.840+0000 7f93937fe640 1 -- 172.21.15.49:0/1928291068 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f93a4076ef0 con 0x7f93ac065d80 2024-03-14T21:13:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.018+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93ac073540 con 0x7f93ac065d80 2024-03-14T21:13:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.018+0000 7f93937fe640 1 -- 172.21.15.49:0/1928291068 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93a407a510 con 0x7f93ac065d80 2024-03-14T21:13:57.002 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-14T21:13:57.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f939806a7a0 msgr2=0x7f939806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f939806a7a0 0x7f939806cc60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93ac071b00 tx=0x7f93a0009040 comp rx=0 tx=0).stop 2024-03-14T21:13:57.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac065d80 msgr2=0x7f93ac1cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac065d80 0x7f93ac1cca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000f80 tx=0x7f93a4004a40 comp rx=0 tx=0).stop 2024-03-14T21:13:57.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 shutdown_connections 2024-03-14T21:13:57.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f939806a7a0 0x7f939806cc60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ac0bb670 0x7f93ac1df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ac0bb290 0x7f93ac1cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 --2- 172.21.15.49:0/1928291068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93ac065d80 0x7f93ac1cca10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 >> 172.21.15.49:0/1928291068 conn(0x7f93ac0b93b0 msgr2=0x7f93ac06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:57.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 shutdown_connections 2024-03-14T21:13:57.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.020+0000 7f93b14ce640 1 -- 172.21.15.49:0/1928291068 wait complete. 2024-03-14T21:13:57.017 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T21:13:57.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.139+0000 7ff704077640 1 Processor -- start 2024-03-14T21:13:57.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.139+0000 7ff704077640 1 -- start start 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff704077640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6fc06a950 0x7ff6fc0bbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff704077640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6fc0bfbc0 con 0x7ff6fc0a5250 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff704077640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6fc0bc950 con 0x7ff6fc06ba90 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff704077640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6fc0bf670 con 0x7ff6fc06a950 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff701e13640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6fc06a950 0x7ff6fc0bbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff702614640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff6fc0a5250 0x7ff6fc155480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55626/0 (socket says 172.21.15.49:55626) 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff701e13640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6fc06a950 0x7ff6fc0bbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36616/0 (socket says 172.21.15.49:36616) 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.140+0000 7ff702614640 1 -- 172.21.15.49:0/3675948086 learned_addr learned my addr 172.21.15.49:0/3675948086 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff701e13640 1 -- 172.21.15.49:0/3675948086 >> v1:172.21.15.97:6789/0 conn(0x7ff6fc06ba90 legacy=0x7ff6fc14ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff700e11640 1 -- 172.21.15.49:0/3675948086 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782024322 0 0) 0x7ff6fc0bfbc0 con 0x7ff6fc0a5250 2024-03-14T21:13:57.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff701e13640 1 -- 172.21.15.49:0/3675948086 >> v1:172.21.15.49:6789/0 conn(0x7ff6fc0a5250 legacy=0x7ff6fc155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff701e13640 1 -- 172.21.15.49:0/3675948086 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc0c0050 con 0x7ff6fc06a950 2024-03-14T21:13:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff701e13640 1 --2- 172.21.15.49:0/3675948086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc06a950 0x7ff6fc0bbd10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00097a0 tx=0x7ff6f002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674d38c9e06338f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff700e11640 1 -- 172.21.15.49:0/3675948086 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f002f050 con 0x7ff6fc06a950 2024-03-14T21:13:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff700e11640 1 -- 172.21.15.49:0/3675948086 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0031700 con 0x7ff6fc06a950 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.141+0000 7ff700e11640 1 -- 172.21.15.49:0/3675948086 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0031a90 con 0x7ff6fc06a950 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.142+0000 7ff704077640 1 -- 172.21.15.49:0/3675948086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc06a950 msgr2=0x7ff6fc0bbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.142+0000 7ff704077640 1 --2- 172.21.15.49:0/3675948086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc06a950 0x7ff6fc0bbd10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00097a0 tx=0x7ff6f002b7a0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.142+0000 7ff704077640 1 -- 172.21.15.49:0/3675948086 shutdown_connections 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.142+0000 7ff704077640 1 --2- 172.21.15.49:0/3675948086 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc06a950 0x7ff6fc0bbd10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.142+0000 7ff704077640 1 -- 172.21.15.49:0/3675948086 >> 172.21.15.49:0/3675948086 conn(0x7ff6fc0c0460 msgr2=0x7ff6fc0c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.142+0000 7ff704077640 1 -- 172.21.15.49:0/3675948086 shutdown_connections 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.142+0000 7ff704077640 1 -- 172.21.15.49:0/3675948086 wait complete. 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.143+0000 7ff704077640 1 Processor -- start 2024-03-14T21:13:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.143+0000 7ff704077640 1 -- start start 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff704077640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6fc06a950 0x7ff6fc141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff704077640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc0a5250 0x7ff6fc1465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff704077640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6fc142b90 0x7ff6fc0b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff704077640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc146b20 con 0x7ff6fc0a5250 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff704077640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0b2670 con 0x7ff6fc06a950 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff704077640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc1441f0 con 0x7ff6fc142b90 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc0a5250 0x7ff6fc1465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc0a5250 0x7ff6fc1465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36632/0 (socket says 172.21.15.49:36632) 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 -- 172.21.15.49:0/2873746492 learned_addr learned my addr 172.21.15.49:0/2873746492 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701e13640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6fc06a950 0x7ff6fc141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff702614640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6fc142b90 0x7ff6fc0b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 -- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6fc142b90 msgr2=0x7ff6fc0b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6fc142b90 0x7ff6fc0b9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 -- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6fc06a950 msgr2=0x7ff6fc141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6fc06a950 0x7ff6fc141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 -- 172.21.15.49:0/2873746492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0009400 con 0x7ff6fc0a5250 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff702614640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6fc142b90 0x7ff6fc0b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.144+0000 7ff701612640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc0a5250 0x7ff6fc1465e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c430 tx=0x7ff6f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.145+0000 7ff6ee7fc640 1 -- 172.21.15.49:0/2873746492 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8011790 con 0x7ff6fc0a5250 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.145+0000 7ff6ee7fc640 1 -- 172.21.15.49:0/2873746492 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f80118f0 con 0x7ff6fc0a5250 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.145+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc144680 con 0x7ff6fc0a5250 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.145+0000 7ff6ee7fc640 1 -- 172.21.15.49:0/2873746492 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8011ce0 con 0x7ff6fc0a5250 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.145+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc143c50 con 0x7ff6fc0a5250 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.146+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc144360 con 0x7ff6fc0a5250 2024-03-14T21:13:57.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.146+0000 7ff6ee7fc640 1 -- 172.21.15.49:0/2873746492 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff6f801c460 con 0x7ff6fc0a5250 2024-03-14T21:13:57.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.146+0000 7ff6ee7fc640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff6e006a7a0 0x7ff6e006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:57.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.146+0000 7ff6ee7fc640 1 -- 172.21.15.49:0/2873746492 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff6f8088810 con 0x7ff6fc0a5250 2024-03-14T21:13:57.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.147+0000 7ff701e13640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff6e006a7a0 0x7ff6e006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:57.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.147+0000 7ff701e13640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff6e006a7a0 0x7ff6e006cc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0035000 tx=0x7ff6f0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:57.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.151+0000 7ff6ee7fc640 1 -- 172.21.15.49:0/2873746492 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff6f8054010 con 0x7ff6fc0a5250 2024-03-14T21:13:57.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.371+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6fc066bb0 con 0x7ff6fc0a5250 2024-03-14T21:13:57.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.371+0000 7ff6ee7fc640 1 -- 172.21.15.49:0/2873746492 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7ff6f8057630 con 0x7ff6fc0a5250 2024-03-14T21:13:57.356 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:57.356 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-14T21:13:57.356 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff6e006a7a0 msgr2=0x7ff6e006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff6e006a7a0 0x7ff6e006cc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0035000 tx=0x7ff6f0008030 comp rx=0 tx=0).stop 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc0a5250 msgr2=0x7ff6fc1465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc0a5250 0x7ff6fc1465e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c430 tx=0x7ff6f800c8f0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 shutdown_connections 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff6e006a7a0 0x7ff6e006cc60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6fc142b90 0x7ff6fc0b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6fc0a5250 0x7ff6fc1465e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 --2- 172.21.15.49:0/2873746492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6fc06a950 0x7ff6fc141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:57.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 >> 172.21.15.49:0/2873746492 conn(0x7ff6fc0c0460 msgr2=0x7ff6fc1d6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:57.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 shutdown_connections 2024-03-14T21:13:57.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:57.373+0000 7ff704077640 1 -- 172.21.15.49:0/2873746492 wait complete. 2024-03-14T21:13:59.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T21:13:59.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b42799640 1 Processor -- start 2024-03-14T21:13:59.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b42799640 1 -- start start 2024-03-14T21:13:59.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b42799640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b3c071540 0x7f0b3c14be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b42799640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0b8c40 con 0x7f0b3c071d10 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b42799640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0b9200 con 0x7f0b3c070550 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b42799640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0709e0 con 0x7f0b3c071540 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b40d36640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0b3c071d10 0x7f0b3c1531a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55638/0 (socket says 172.21.15.49:55638) 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.503+0000 7f0b40d36640 1 -- 172.21.15.49:0/3785719632 learned_addr learned my addr 172.21.15.49:0/3785719632 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/3785719632 >> v2:172.21.15.97:3300/0 conn(0x7f0b3c071540 0x7f0b3c14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336886843 0 0) 0x7f0b3c0b8c40 con 0x7f0b3c071d10 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b24003520 con 0x7f0b3c071d10 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370473373 0 0) 0x7f0b24003520 con 0x7f0b3c071d10 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 >> v2:172.21.15.97:3300/0 conn(0x7f0b3c071540 msgr2=0x7f0b3c14be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 --2- 172.21.15.49:0/3785719632 >> v2:172.21.15.97:3300/0 conn(0x7f0b3c071540 0x7f0b3c14be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 >> v1:172.21.15.97:6789/0 conn(0x7f0b3c070550 legacy=0x7f0b3c1427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0b8e90 con 0x7f0b3c071d10 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f0b340019f0 con 0x7f0b3c071d10 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 >> v1:172.21.15.49:6789/0 conn(0x7f0b3c071d10 legacy=0x7f0b3c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24003970 0x7f0b24003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24004370 0x7f0b24004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b24005040 0x7f0b2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0709e0 con 0x7f0b24003970 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2400bef0 con 0x7f0b24004370 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2400c0b0 con 0x7f0b24005040 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24003970 0x7f0b24003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b3b7fe640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24004370 0x7f0b24004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.504+0000 7f0b40d36640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b24005040 0x7f0b2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3bfff640 1 -- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b24005040 msgr2=0x7f0b2400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b24005040 0x7f0b2400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3bfff640 1 -- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24004370 msgr2=0x7f0b24004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:59.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24004370 0x7f0b24004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3bfff640 1 -- 172.21.15.49:0/3785719632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b24003520 con 0x7f0b24003970 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b40d36640 1 -- 172.21.15.49:0/3785719632 reap_dead start 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24003970 0x7f0b24003d50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009aa0 tx=0x7f0b240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f44aaf34d54b8ac server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c02c6b0 con 0x7f0b24003970 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.505+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c02c810 con 0x7f0b24003970 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b3affd640 1 -- 172.21.15.49:0/3785719632 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c02cbc0 con 0x7f0b24003970 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b42799640 1 -- 172.21.15.49:0/3785719632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24003970 msgr2=0x7f0b24003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b42799640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24003970 0x7f0b24003d50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009aa0 tx=0x7f0b240057f0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b42799640 1 -- 172.21.15.49:0/3785719632 shutdown_connections 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b42799640 1 --2- 172.21.15.49:0/3785719632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b24003970 0x7f0b24003d50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b42799640 1 -- 172.21.15.49:0/3785719632 >> 172.21.15.49:0/3785719632 conn(0x7f0b3c0b9d30 msgr2=0x7f0b3c0ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b42799640 1 -- 172.21.15.49:0/3785719632 shutdown_connections 2024-03-14T21:13:59.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.506+0000 7f0b42799640 1 -- 172.21.15.49:0/3785719632 wait complete. 2024-03-14T21:13:59.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.507+0000 7f0b42799640 1 Processor -- start 2024-03-14T21:13:59.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.507+0000 7f0b42799640 1 -- start start 2024-03-14T21:13:59.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b42799640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24003970 0x7f0b3c1e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b42799640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c0b6d10 0x7f0b3c0bc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b42799640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b3c13b1a0 0x7f0b3c13b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b42799640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0bc960 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b42799640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c13bb40 con 0x7f0b24003970 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b42799640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0bced0 con 0x7f0b3c13b1a0 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c0b6d10 0x7f0b3c0bc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c0b6d10 0x7f0b3c0bc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36640/0 (socket says 172.21.15.49:36640) 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 -- 172.21.15.49:0/22777630 learned_addr learned my addr 172.21.15.49:0/22777630 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24003970 0x7f0b3c1e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b40d36640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b3c13b1a0 0x7f0b3c13b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 -- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b3c13b1a0 msgr2=0x7f0b3c13b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b3c13b1a0 0x7f0b3c13b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 -- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24003970 msgr2=0x7f0b3c1e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24003970 0x7f0b3c1e0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.508+0000 7f0b3b7fe640 1 -- 172.21.15.49:0/22777630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0b6880 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.509+0000 7f0b3b7fe640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c0b6d10 0x7f0b3c0bc420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000f780 tx=0x7f0b3000fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.509+0000 7f0b38ff9640 1 -- 172.21.15.49:0/22777630 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300050d0 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.509+0000 7f0b38ff9640 1 -- 172.21.15.49:0/22777630 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b30005250 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.509+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c0b5ac0 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.509+0000 7f0b38ff9640 1 -- 172.21.15.49:0/22777630 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000d6d0 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.509+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c0b4e20 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.510+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c0b6a10 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.510+0000 7f0b38ff9640 1 -- 172.21.15.49:0/22777630 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0b3000d830 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.511+0000 7f0b38ff9640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0b2806a7a0 0x7f0b2806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.511+0000 7f0b38ff9640 1 -- 172.21.15.49:0/22777630 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0b300887c0 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.511+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0b2806a7a0 0x7f0b2806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.511+0000 7f0b3bfff640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0b2806a7a0 0x7f0b2806cc60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c036000 tx=0x7f0b2c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:59.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.515+0000 7f0b38ff9640 1 -- 172.21.15.49:0/22777630 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b30053fc0 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.732+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b3c073540 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.732+0000 7f0b38ff9640 1 -- 172.21.15.49:0/22777630 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2765 (secure 0 0 0) 0x7f0b300575e0 con 0x7f0b3c0b6d10 2024-03-14T21:13:59.717 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:13:59.717 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2343109927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2343109927},{"type":"v1","addr":"172.21.15.97:6835","nonce":2343109927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1491968682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1491968682},{"type":"v1","addr":"172.21.15.49:6833","nonce":1491968682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-14T21:13:59.717 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-03-14T21:13:59.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0b2806a7a0 msgr2=0x7f0b2806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0b2806a7a0 0x7f0b2806cc60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c036000 tx=0x7f0b2c030000 comp rx=0 tx=0).stop 2024-03-14T21:13:59.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c0b6d10 msgr2=0x7f0b3c0bc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c0b6d10 0x7f0b3c0bc420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000f780 tx=0x7f0b3000fb80 comp rx=0 tx=0).stop 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 shutdown_connections 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0b2806a7a0 0x7f0b2806cc60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b3c13b1a0 0x7f0b3c13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b3c0b6d10 0x7f0b3c0bc420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.734+0000 7f0b42799640 1 --2- 172.21.15.49:0/22777630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b24003970 0x7f0b3c1e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.735+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 >> 172.21.15.49:0/22777630 conn(0x7f0b3c0b9d30 msgr2=0x7f0b3c14be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.735+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 shutdown_connections 2024-03-14T21:13:59.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.735+0000 7f0b42799640 1 -- 172.21.15.49:0/22777630 wait complete. 2024-03-14T21:13:59.729 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:13:59.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.853+0000 7f8006bae640 1 Processor -- start 2024-03-14T21:13:59.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.853+0000 7f8006bae640 1 -- start start 2024-03-14T21:13:59.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f8006bae640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f80000bbd10 0x7f8000156490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f8006bae640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80000bfbc0 con 0x7f800006ba90 2024-03-14T21:13:59.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f8006bae640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80000bc950 con 0x7f800006a950 2024-03-14T21:13:59.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f8006bae640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80000bf670 con 0x7f80000bbd10 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f800514b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f80000bbd10 0x7f8000156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f800514b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f80000bbd10 0x7f8000156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36656/0 (socket says 172.21.15.49:36656) 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f800514b640 1 -- 172.21.15.49:0/816427078 learned_addr learned my addr 172.21.15.49:0/816427078 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f7ffffff640 1 --1- 172.21.15.49:0/816427078 >> v1:172.21.15.49:6789/0 conn(0x7f800006ba90 0x7f80000a5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55650/0 (socket says 172.21.15.49:55650) 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f800514b640 1 -- 172.21.15.49:0/816427078 >> v1:172.21.15.97:6789/0 conn(0x7f800006a950 legacy=0x7f800014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f800514b640 1 -- 172.21.15.49:0/816427078 >> v1:172.21.15.49:6789/0 conn(0x7f800006ba90 legacy=0x7f80000a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f800514b640 1 -- 172.21.15.49:0/816427078 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000c0050 con 0x7f80000bbd10 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.854+0000 7f7fff7fe640 1 -- 172.21.15.49:0/816427078 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219911342 0 0) 0x7f80000bfbc0 con 0x7f800006ba90 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.855+0000 7f800514b640 1 --2- 172.21.15.49:0/816427078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f8000156490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009e50 tx=0x7f7ff802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155066e54de5865b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.855+0000 7f7fff7fe640 1 -- 172.21.15.49:0/816427078 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff802c9f0 con 0x7f80000bbd10 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.855+0000 7f7fff7fe640 1 -- 172.21.15.49:0/816427078 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff802cb50 con 0x7f80000bbd10 2024-03-14T21:13:59.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.855+0000 7f7fff7fe640 1 -- 172.21.15.49:0/816427078 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80343f0 con 0x7f80000bbd10 2024-03-14T21:13:59.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.856+0000 7f8006bae640 1 -- 172.21.15.49:0/816427078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 msgr2=0x7f8000156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.856+0000 7f8006bae640 1 --2- 172.21.15.49:0/816427078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f8000156490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009e50 tx=0x7f7ff802be50 comp rx=0 tx=0).stop 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.856+0000 7f8006bae640 1 -- 172.21.15.49:0/816427078 shutdown_connections 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.856+0000 7f8006bae640 1 --2- 172.21.15.49:0/816427078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f8000156490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.856+0000 7f8006bae640 1 -- 172.21.15.49:0/816427078 >> 172.21.15.49:0/816427078 conn(0x7f80000c0460 msgr2=0x7f80000c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.856+0000 7f8006bae640 1 -- 172.21.15.49:0/816427078 shutdown_connections 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.856+0000 7f8006bae640 1 -- 172.21.15.49:0/816427078 wait complete. 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.857+0000 7f8006bae640 1 Processor -- start 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.857+0000 7f8006bae640 1 -- start start 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f8006bae640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f800006a950 0x7f80001e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f8006bae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f800009d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f8006bae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80001e4730 0x7f8000149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f8006bae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80001e3a80 con 0x7f80000bbd10 2024-03-14T21:13:59.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f8006bae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800009b9b0 con 0x7f80001e4730 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f8006bae640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800009ee00 con 0x7f800006a950 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f800009d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f800009d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36658/0 (socket says 172.21.15.49:36658) 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 -- 172.21.15.49:0/2080000087 learned_addr learned my addr 172.21.15.49:0/2080000087 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f800514b640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80001e4730 0x7f8000149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f800494a640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f800006a950 0x7f80001e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 -- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f800006a950 msgr2=0x7f80001e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f800006a950 0x7f80001e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 -- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80001e4730 msgr2=0x7f8000149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80001e4730 0x7f8000149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 -- 172.21.15.49:0/2080000087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff8009b30 con 0x7f80000bbd10 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffffff640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f800009d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c4d0 tx=0x7f7ff400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.858+0000 7f7ffd7fa640 1 -- 172.21.15.49:0/2080000087 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40074e0 con 0x7f80000bbd10 2024-03-14T21:13:59.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.859+0000 7f7ffd7fa640 1 -- 172.21.15.49:0/2080000087 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff4007660 con 0x7f80000bbd10 2024-03-14T21:13:59.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.859+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800009f600 con 0x7f80000bbd10 2024-03-14T21:13:59.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.859+0000 7f7ffd7fa640 1 -- 172.21.15.49:0/2080000087 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4004a20 con 0x7f80000bbd10 2024-03-14T21:13:59.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.859+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800009f220 con 0x7f80000bbd10 2024-03-14T21:13:59.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.859+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80001e3f10 con 0x7f80000bbd10 2024-03-14T21:13:59.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.860+0000 7f7ffd7fa640 1 -- 172.21.15.49:0/2080000087 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7ff4004b80 con 0x7f80000bbd10 2024-03-14T21:13:59.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.860+0000 7f7ffd7fa640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7fec06a7d0 0x7f7fec06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:13:59.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.861+0000 7f7ffd7fa640 1 -- 172.21.15.49:0/2080000087 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ff4087c30 con 0x7f80000bbd10 2024-03-14T21:13:59.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.861+0000 7f800494a640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7fec06a7d0 0x7f7fec06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:13:59.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.861+0000 7f800494a640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7fec06a7d0 0x7f7fec06cc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0006fd0 tx=0x7f7ff0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:13:59.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:13:59.865+0000 7f7ffd7fa640 1 -- 172.21.15.49:0/2080000087 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f80001e3f10 con 0x7f80000bbd10 2024-03-14T21:14:00.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.045+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80001e3f10 con 0x7f80000bbd10 2024-03-14T21:14:00.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.046+0000 7f7ffd7fa640 1 -- 172.21.15.49:0/2080000087 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f80001e3f10 con 0x7f80000bbd10 2024-03-14T21:14:00.030 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:00.030 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:13:53.613681+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7fec06a7d0 msgr2=0x7f7fec06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7fec06a7d0 0x7f7fec06cc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0006fd0 tx=0x7f7ff0008040 comp rx=0 tx=0).stop 2024-03-14T21:14:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 msgr2=0x7f800009d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f800009d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c4d0 tx=0x7f7ff400c990 comp rx=0 tx=0).stop 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 shutdown_connections 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7fec06a7d0 0x7f7fec06cc90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80001e4730 0x7f8000149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80000bbd10 0x7f800009d3f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 --2- 172.21.15.49:0/2080000087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f800006a950 0x7f80001e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 >> 172.21.15.49:0/2080000087 conn(0x7f80000c0460 msgr2=0x7f80000b7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 shutdown_connections 2024-03-14T21:14:00.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.048+0000 7f8006bae640 1 -- 172.21.15.49:0/2080000087 wait complete. 2024-03-14T21:14:00.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T21:13:50.964908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T21:14:00.043 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T21:14:00.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.167+0000 7f56a968b640 1 Processor -- start 2024-03-14T21:14:00.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.167+0000 7f56a968b640 1 -- start start 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a968b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a414b660 0x7f56a406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a968b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56a40b9080 0x7f56a4155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a968b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a406e540 con 0x7f56a40b8c40 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a968b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a414e090 con 0x7f56a40b9080 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a968b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a40b9970 con 0x7f56a414b660 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56a40b9080 0x7f56a4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56a40b9080 0x7f56a4155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36664/0 (socket says 172.21.15.49:36664) 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a37fe640 1 -- 172.21.15.49:0/305007477 learned_addr learned my addr 172.21.15.49:0/305007477 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.168+0000 7f56a2ffd640 1 --2- 172.21.15.49:0/305007477 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a414b660 0x7f56a406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.169+0000 7f56a37fe640 1 -- 172.21.15.49:0/305007477 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a414b660 msgr2=0x7f56a406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:00.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.169+0000 7f56a37fe640 1 --2- 172.21.15.49:0/305007477 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a414b660 0x7f56a406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.169+0000 7f56a37fe640 1 -- 172.21.15.49:0/305007477 >> v1:172.21.15.97:6789/0 conn(0x7f56a40b8c40 legacy=0x7f56a414e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.169+0000 7f56a37fe640 1 -- 172.21.15.49:0/305007477 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a40703c0 con 0x7f56a40b9080 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.169+0000 7f56a37fe640 1 --2- 172.21.15.49:0/305007477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a4155380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f569c009e50 tx=0x7f569c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0a102ecd322df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a27fc640 1 -- 172.21.15.49:0/305007477 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c02cb90 con 0x7f56a40b9080 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a27fc640 1 -- 172.21.15.49:0/305007477 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c02ccf0 con 0x7f56a40b9080 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a27fc640 1 -- 172.21.15.49:0/305007477 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c034610 con 0x7f56a40b9080 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 -- 172.21.15.49:0/305007477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 msgr2=0x7f56a4155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 --2- 172.21.15.49:0/305007477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a4155380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f569c009e50 tx=0x7f569c02bff0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 -- 172.21.15.49:0/305007477 shutdown_connections 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 --2- 172.21.15.49:0/305007477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a4155380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 --2- 172.21.15.49:0/305007477 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a414b660 0x7f56a406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 -- 172.21.15.49:0/305007477 >> 172.21.15.49:0/305007477 conn(0x7f56a40b9d30 msgr2=0x7f56a40ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:00.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 -- 172.21.15.49:0/305007477 shutdown_connections 2024-03-14T21:14:00.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.170+0000 7f56a968b640 1 -- 172.21.15.49:0/305007477 wait complete. 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.171+0000 7f56a968b640 1 Processor -- start 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.171+0000 7f56a968b640 1 -- start start 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a968b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a40b8c40 0x7f56a41453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a968b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a41475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a968b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56a414b660 0x7f56a4146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a968b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41458e0 con 0x7f56a40b9080 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a968b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41d7d00 con 0x7f56a414b660 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a968b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a409ed20 con 0x7f56a40b8c40 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a41475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a41475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36678/0 (socket says 172.21.15.49:36678) 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 -- 172.21.15.49:0/900372351 learned_addr learned my addr 172.21.15.49:0/900372351 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 -- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a40b8c40 msgr2=0x7f56a41453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.158 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a2ffd640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a40b8c40 0x7f56a41453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a40b8c40 0x7f56a41453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f56a37fe640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56a414b660 0x7f56a4146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 -- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56a414b660 msgr2=0x7f56a4146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56a414b660 0x7f56a4146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 -- 172.21.15.49:0/900372351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c009b30 con 0x7f56a40b9080 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.172+0000 7f569bfff640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a41475c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c4d0 tx=0x7f568c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.173+0000 7f569b7fe640 1 -- 172.21.15.49:0/900372351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c0052d0 con 0x7f56a40b9080 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.173+0000 7f569b7fe640 1 -- 172.21.15.49:0/900372351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c005430 con 0x7f56a40b9080 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.173+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a4147df0 con 0x7f56a40b9080 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.173+0000 7f569b7fe640 1 -- 172.21.15.49:0/900372351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c012850 con 0x7f56a40b9080 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.173+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a4145be0 con 0x7f56a40b9080 2024-03-14T21:14:00.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.174+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a40b96e0 con 0x7f56a40b9080 2024-03-14T21:14:00.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.174+0000 7f569b7fe640 1 -- 172.21.15.49:0/900372351 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f568c0129b0 con 0x7f56a40b9080 2024-03-14T21:14:00.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.174+0000 7f569b7fe640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f568406a7d0 0x7f568406cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.174+0000 7f569b7fe640 1 -- 172.21.15.49:0/900372351 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f568c088550 con 0x7f56a40b9080 2024-03-14T21:14:00.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.175+0000 7f56a2ffd640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f568406a7d0 0x7f568406cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.175+0000 7f56a2ffd640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f568406a7d0 0x7f568406cc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5694009970 tx=0x7f5694008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.179+0000 7f569b7fe640 1 -- 172.21.15.49:0/900372351 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f568c053cc0 con 0x7f56a40b9080 2024-03-14T21:14:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.356+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56a406e9c0 con 0x7f56a40b9080 2024-03-14T21:14:00.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.357+0000 7f569b7fe640 1 -- 172.21.15.49:0/900372351 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f568c0572e0 con 0x7f56a40b9080 2024-03-14T21:14:00.341 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f568406a7d0 msgr2=0x7f568406cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f568406a7d0 0x7f568406cc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5694009970 tx=0x7f5694008040 comp rx=0 tx=0).stop 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 msgr2=0x7f56a41475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a41475c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c4d0 tx=0x7f568c00c990 comp rx=0 tx=0).stop 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 shutdown_connections 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f568406a7d0 0x7f568406cc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56a414b660 0x7f56a4146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56a40b9080 0x7f56a41475c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 --2- 172.21.15.49:0/900372351 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56a40b8c40 0x7f56a41453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 >> 172.21.15.49:0/900372351 conn(0x7f56a40b9d30 msgr2=0x7f56a40bb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:00.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 shutdown_connections 2024-03-14T21:14:00.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.359+0000 7f56a968b640 1 -- 172.21.15.49:0/900372351 wait complete. 2024-03-14T21:14:00.353 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T21:14:00.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.475+0000 7f39003b1640 1 Processor -- start 2024-03-14T21:14:00.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.475+0000 7f39003b1640 1 -- start start 2024-03-14T21:14:00.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f39003b1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38f80bb1c0 0x7f38f8156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f39003b1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80b3a20 con 0x7f38f80bad60 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f39003b1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f806adf0 con 0x7f38f814b820 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f39003b1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80bb5f0 con 0x7f38f80bb1c0 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f38fe94e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38f80bb1c0 0x7f38f8156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f38fe94e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38f80bb1c0 0x7f38f8156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37232/0 (socket says 172.21.15.49:37232) 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f38fe94e640 1 -- 172.21.15.49:0/1814583948 learned_addr learned my addr 172.21.15.49:0/1814583948 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f38fe94e640 1 -- 172.21.15.49:0/1814583948 >> v1:172.21.15.97:6789/0 conn(0x7f38f814b820 legacy=0x7f38f806a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f38fe94e640 1 -- 172.21.15.49:0/1814583948 >> v1:172.21.15.49:6789/0 conn(0x7f38f80bad60 legacy=0x7f38f8153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f38fd14b640 1 -- 172.21.15.49:0/1814583948 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406814197 0 0) 0x7f38f80b3a20 con 0x7f38f80bad60 2024-03-14T21:14:00.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.476+0000 7f38fe94e640 1 -- 172.21.15.49:0/1814583948 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80ba550 con 0x7f38f80bb1c0 2024-03-14T21:14:00.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.477+0000 7f38fe94e640 1 --2- 172.21.15.49:0/1814583948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f80bb1c0 0x7f38f8156cc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e50 tx=0x7f38f002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d44fbdc02e5173 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.477+0000 7f38fd14b640 1 -- 172.21.15.49:0/1814583948 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f002cb20 con 0x7f38f80bb1c0 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.477+0000 7f38fd14b640 1 -- 172.21.15.49:0/1814583948 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f002cc80 con 0x7f38f80bb1c0 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.477+0000 7f38fd14b640 1 -- 172.21.15.49:0/1814583948 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00343f0 con 0x7f38f80bb1c0 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.478+0000 7f39003b1640 1 -- 172.21.15.49:0/1814583948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f80bb1c0 msgr2=0x7f38f8156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.478+0000 7f39003b1640 1 --2- 172.21.15.49:0/1814583948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f80bb1c0 0x7f38f8156cc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e50 tx=0x7f38f002bf80 comp rx=0 tx=0).stop 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.478+0000 7f39003b1640 1 -- 172.21.15.49:0/1814583948 shutdown_connections 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.478+0000 7f39003b1640 1 --2- 172.21.15.49:0/1814583948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f80bb1c0 0x7f38f8156cc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.478+0000 7f39003b1640 1 -- 172.21.15.49:0/1814583948 >> 172.21.15.49:0/1814583948 conn(0x7f38f80bcd80 msgr2=0x7f38f80bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.478+0000 7f39003b1640 1 -- 172.21.15.49:0/1814583948 shutdown_connections 2024-03-14T21:14:00.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.478+0000 7f39003b1640 1 -- 172.21.15.49:0/1814583948 wait complete. 2024-03-14T21:14:00.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.479+0000 7f39003b1640 1 Processor -- start 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.479+0000 7f39003b1640 1 -- start start 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f80bad60 0x7f38f8147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38f80bb1c0 0x7f38f8148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f814b820 0x7f38f809e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8149740 con 0x7f38f814b820 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8147ab0 con 0x7f38f80bad60 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f81e0440 con 0x7f38f80bb1c0 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f814b820 0x7f38f809e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f814b820 0x7f38f809e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37238/0 (socket says 172.21.15.49:37238) 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 -- 172.21.15.49:0/2458532677 learned_addr learned my addr 172.21.15.49:0/2458532677 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fd94c640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38f80bb1c0 0x7f38f8148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe14d640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f80bad60 0x7f38f8147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 -- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38f80bb1c0 msgr2=0x7f38f8148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38f80bb1c0 0x7f38f8148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 -- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f80bad60 msgr2=0x7f38f8147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f80bad60 0x7f38f8147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 -- 172.21.15.49:0/2458532677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0009b30 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe14d640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f80bad60 0x7f38f8147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38fe94e640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f814b820 0x7f38f809e470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38f0036040 tx=0x7f38f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38eeffd640 1 -- 172.21.15.49:0/2458532677 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0009510 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38eeffd640 1 -- 172.21.15.49:0/2458532677 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f0007e10 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f81dfb10 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f38eeffd640 1 -- 172.21.15.49:0/2458532677 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f003e670 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.480+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f81e00d0 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.481+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f81e05f0 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.482+0000 7f38eeffd640 1 -- 172.21.15.49:0/2458532677 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f38f003e7d0 con 0x7f38f814b820 2024-03-14T21:14:00.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.482+0000 7f38eeffd640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f38dc06a7d0 0x7f38dc06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.482+0000 7f38eeffd640 1 -- 172.21.15.49:0/2458532677 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38f00ab330 con 0x7f38f814b820 2024-03-14T21:14:00.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.482+0000 7f38fe14d640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f38dc06a7d0 0x7f38dc06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.483+0000 7f38fe14d640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f38dc06a7d0 0x7f38dc06cc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38f400a830 tx=0x7f38f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.487+0000 7f38eeffd640 1 -- 172.21.15.49:0/2458532677 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f38f81e05f0 con 0x7f38f814b820 2024-03-14T21:14:00.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.673+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f38f81e05f0 con 0x7f38f814b820 2024-03-14T21:14:00.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.673+0000 7f38eeffd640 1 -- 172.21.15.49:0/2458532677 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f38f81e05f0 con 0x7f38f814b820 2024-03-14T21:14:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:00.658 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-03-14T21:14:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-14T21:14:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-03-14T21:14:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-03-14T21:14:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-14T21:14:00.658 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f38dc06a7d0 msgr2=0x7f38dc06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f38dc06a7d0 0x7f38dc06cc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38f400a830 tx=0x7f38f4008040 comp rx=0 tx=0).stop 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f814b820 msgr2=0x7f38f809e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f814b820 0x7f38f809e470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38f0036040 tx=0x7f38f0004a40 comp rx=0 tx=0).stop 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 shutdown_connections 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f38dc06a7d0 0x7f38dc06cc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38f814b820 0x7f38f809e470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38f80bb1c0 0x7f38f8148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 --2- 172.21.15.49:0/2458532677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f80bad60 0x7f38f8147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 >> 172.21.15.49:0/2458532677 conn(0x7f38f80bcd80 msgr2=0x7f38f806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:00.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 shutdown_connections 2024-03-14T21:14:00.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.675+0000 7f39003b1640 1 -- 172.21.15.49:0/2458532677 wait complete. 2024-03-14T21:14:00.670 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T21:14:00.670 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T21:14:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.795+0000 7fb72e438640 1 Processor -- start 2024-03-14T21:14:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.795+0000 7fb72e438640 1 -- start start 2024-03-14T21:14:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.796+0000 7fb72e438640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bfc80 0x7fb72814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.796+0000 7fb72e438640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72806ae40 con 0x7fb7280bc940 2024-03-14T21:14:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.796+0000 7fb72e438640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7280bc590 con 0x7fb7280bb010 2024-03-14T21:14:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.796+0000 7fb72e438640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7280ba750 con 0x7fb7280bfc80 2024-03-14T21:14:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.796+0000 7fb7277fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb7280bc940 0x7fb7280b9c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36782/0 (socket says 172.21.15.49:36782) 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.796+0000 7fb7277fe640 1 -- 172.21.15.49:0/959311887 learned_addr learned my addr 172.21.15.49:0/959311887 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.796+0000 7fb727fff640 1 --2- 172.21.15.49:0/959311887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bfc80 0x7fb72814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.797+0000 7fb726ffd640 1 -- 172.21.15.49:0/959311887 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743218793 0 0) 0x7fb72806ae40 con 0x7fb7280bc940 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.797+0000 7fb726ffd640 1 -- 172.21.15.49:0/959311887 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb710003520 con 0x7fb7280bc940 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.797+0000 7fb727fff640 1 -- 172.21.15.49:0/959311887 >> v1:172.21.15.97:6789/0 conn(0x7fb7280bb010 legacy=0x7fb7281555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.797+0000 7fb726ffd640 1 -- 172.21.15.49:0/959311887 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106261339 0 0) 0x7fb7280bc590 con 0x7fb7280bb010 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.797+0000 7fb727fff640 1 -- 172.21.15.49:0/959311887 >> v1:172.21.15.49:6789/0 conn(0x7fb7280bc940 legacy=0x7fb7280b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.797+0000 7fb727fff640 1 -- 172.21.15.49:0/959311887 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7280bb4c0 con 0x7fb7280bfc80 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb727fff640 1 --2- 172.21.15.49:0/959311887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bfc80 0x7fb72814ad90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7180097d0 tx=0x7fb71802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56af1abba94d4ddc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb726ffd640 1 -- 172.21.15.49:0/959311887 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718031770 con 0x7fb7280bfc80 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb726ffd640 1 -- 172.21.15.49:0/959311887 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7180318d0 con 0x7fb7280bfc80 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb726ffd640 1 -- 172.21.15.49:0/959311887 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718031c60 con 0x7fb7280bfc80 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb72e438640 1 -- 172.21.15.49:0/959311887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bfc80 msgr2=0x7fb72814ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb72e438640 1 --2- 172.21.15.49:0/959311887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bfc80 0x7fb72814ad90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7180097d0 tx=0x7fb71802b810 comp rx=0 tx=0).stop 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb72e438640 1 -- 172.21.15.49:0/959311887 shutdown_connections 2024-03-14T21:14:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb72e438640 1 --2- 172.21.15.49:0/959311887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bfc80 0x7fb72814ad90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb72e438640 1 -- 172.21.15.49:0/959311887 >> 172.21.15.49:0/959311887 conn(0x7fb7280bcda0 msgr2=0x7fb7280bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb72e438640 1 -- 172.21.15.49:0/959311887 shutdown_connections 2024-03-14T21:14:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.798+0000 7fb72e438640 1 -- 172.21.15.49:0/959311887 wait complete. 2024-03-14T21:14:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.799+0000 7fb72e438640 1 Processor -- start 2024-03-14T21:14:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72e438640 1 -- start start 2024-03-14T21:14:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72e438640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bb010 0x7fb7281d5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72e438640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7280bc940 0x7fb7281e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72e438640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7280bfc80 0x7fb7281e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72e438640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7280a6190 con 0x7fb7280bfc80 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72e438640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb728099630 con 0x7fb7280bc940 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72e438640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb728099910 con 0x7fb7280bb010 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72c9d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7280bfc80 0x7fb7281e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72c9d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7280bfc80 0x7fb7281e07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37244/0 (socket says 172.21.15.49:37244) 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb72c9d5640 1 -- 172.21.15.49:0/2422534373 learned_addr learned my addr 172.21.15.49:0/2422534373 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb727fff640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bb010 0x7fb7281d5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.800+0000 7fb7277fe640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7280bc940 0x7fb7281e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72c9d5640 1 -- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bb010 msgr2=0x7fb7281d5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72c9d5640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bb010 0x7fb7281d5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72c9d5640 1 -- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7280bc940 msgr2=0x7fb7281e02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72c9d5640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7280bc940 0x7fb7281e02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72c9d5640 1 -- 172.21.15.49:0/2422534373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb718009400 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb727fff640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bb010 0x7fb7281d5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb7277fe640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7280bc940 0x7fb7281e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72c9d5640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7280bfc80 0x7fb7281e07e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb72000c710 tx=0x7fb72000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb724ff9640 1 -- 172.21.15.49:0/2422534373 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720003da0 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb724ff9640 1 -- 172.21.15.49:0/2422534373 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72000ce20 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb728099c80 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb724ff9640 1 -- 172.21.15.49:0/2422534373 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720005290 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.801+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7280a5d70 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.802+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb728099ad0 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.803+0000 7fb724ff9640 1 -- 172.21.15.49:0/2422534373 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb7200053f0 con 0x7fb7280bfc80 2024-03-14T21:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.803+0000 7fb724ff9640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb71406a7a0 0x7fb71406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.803+0000 7fb724ff9640 1 -- 172.21.15.49:0/2422534373 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb720088220 con 0x7fb7280bfc80 2024-03-14T21:14:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.803+0000 7fb727fff640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb71406a7a0 0x7fb71406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.804+0000 7fb727fff640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb71406a7a0 0x7fb71406cc60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb718036000 tx=0x7fb718008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:00.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.807+0000 7fb724ff9640 1 -- 172.21.15.49:0/2422534373 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb720053a20 con 0x7fb7280bfc80 2024-03-14T21:14:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:00.988+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb7280a62d0 con 0x7fb7280bfc80 2024-03-14T21:14:01.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.639+0000 7fb724ff9640 1 -- 172.21.15.49:0/2422534373 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb7280a62d0 con 0x7fb7280bfc80 2024-03-14T21:14:01.623 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb71406a7a0 msgr2=0x7fb71406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb71406a7a0 0x7fb71406cc60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb718036000 tx=0x7fb718008030 comp rx=0 tx=0).stop 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7280bfc80 msgr2=0x7fb7281e07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7280bfc80 0x7fb7281e07e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb72000c710 tx=0x7fb72000cbd0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 shutdown_connections 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb71406a7a0 0x7fb71406cc60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7280bfc80 0x7fb7281e07e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7280bc940 0x7fb7281e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 --2- 172.21.15.49:0/2422534373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7280bb010 0x7fb7281d5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 >> 172.21.15.49:0/2422534373 conn(0x7fb7280bcda0 msgr2=0x7fb7281d6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 shutdown_connections 2024-03-14T21:14:01.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.642+0000 7fb72e438640 1 -- 172.21.15.49:0/2422534373 wait complete. 2024-03-14T21:14:01.641 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.770+0000 7f263e48b640 1 Processor -- start 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f263e48b640 1 -- start start 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f263e48b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26380bf9c0 0x7f263814ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f263e48b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bfe50 0x7f2638155670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f263e48b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380bac50 con 0x7f26380b9f70 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f263e48b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263806ace0 con 0x7f26380bf9c0 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f263e48b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380baea0 con 0x7f26380bfe50 2024-03-14T21:14:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f26377fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26380bf9c0 0x7f263814ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f26377fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26380bf9c0 0x7f263814ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37252/0 (socket says 172.21.15.49:37252) 2024-03-14T21:14:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.771+0000 7f26377fe640 1 -- 172.21.15.49:0/2779521080 learned_addr learned my addr 172.21.15.49:0/2779521080 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.772+0000 7f263ca28640 1 --2- 172.21.15.49:0/2779521080 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bfe50 0x7f2638155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.772+0000 7f26377fe640 1 -- 172.21.15.49:0/2779521080 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bfe50 msgr2=0x7f2638155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.772+0000 7f26377fe640 1 --2- 172.21.15.49:0/2779521080 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bfe50 0x7f2638155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.772+0000 7f26377fe640 1 -- 172.21.15.49:0/2779521080 >> v1:172.21.15.97:6789/0 conn(0x7f26380b9f70 legacy=0x7f263806a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.772+0000 7f26377fe640 1 -- 172.21.15.49:0/2779521080 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380bb180 con 0x7f26380bf9c0 2024-03-14T21:14:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.772+0000 7f26377fe640 1 --2- 172.21.15.49:0/2779521080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380bf9c0 0x7f263814ae40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f262c009870 tx=0x7f262c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9af12a4c329a4f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f2636ffd640 1 -- 172.21.15.49:0/2779521080 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c030840 con 0x7f26380bf9c0 2024-03-14T21:14:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f2636ffd640 1 -- 172.21.15.49:0/2779521080 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c0309a0 con 0x7f26380bf9c0 2024-03-14T21:14:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f2636ffd640 1 -- 172.21.15.49:0/2779521080 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c030d30 con 0x7f26380bf9c0 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 -- 172.21.15.49:0/2779521080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380bf9c0 msgr2=0x7f263814ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 --2- 172.21.15.49:0/2779521080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380bf9c0 0x7f263814ae40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f262c009870 tx=0x7f262c02b980 comp rx=0 tx=0).stop 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 -- 172.21.15.49:0/2779521080 shutdown_connections 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 --2- 172.21.15.49:0/2779521080 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bfe50 0x7f2638155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 --2- 172.21.15.49:0/2779521080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380bf9c0 0x7f263814ae40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 -- 172.21.15.49:0/2779521080 >> 172.21.15.49:0/2779521080 conn(0x7f26380c02d0 msgr2=0x7f26380c06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 -- 172.21.15.49:0/2779521080 shutdown_connections 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.773+0000 7f263e48b640 1 -- 172.21.15.49:0/2779521080 wait complete. 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.774+0000 7f263e48b640 1 Processor -- start 2024-03-14T21:14:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.774+0000 7f263e48b640 1 -- start start 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f263e48b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380b9f70 0x7f26381e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f263e48b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bf9c0 0x7f26380a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f263e48b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26380bfe50 0x7f26380b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f263e48b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380a3890 con 0x7f26380b9f70 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f263e48b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380a3ae0 con 0x7f26380bfe50 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f263e48b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380a3dc0 con 0x7f26380bf9c0 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f2637fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380b9f70 0x7f26381e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f2637fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380b9f70 0x7f26381e43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37262/0 (socket says 172.21.15.49:37262) 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f2637fff640 1 -- 172.21.15.49:0/4009796284 learned_addr learned my addr 172.21.15.49:0/4009796284 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f263ca28640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26380bfe50 0x7f26380b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f26377fe640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bf9c0 0x7f26380a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f2637fff640 1 -- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bf9c0 msgr2=0x7f26380a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f2637fff640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bf9c0 0x7f26380a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f2637fff640 1 -- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26380bfe50 msgr2=0x7f26380b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.775+0000 7f2637fff640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26380bfe50 0x7f26380b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.776+0000 7f2637fff640 1 -- 172.21.15.49:0/4009796284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094d0 con 0x7f26380b9f70 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.776+0000 7f2637fff640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380b9f70 0x7f26381e43e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f262800c400 tx=0x7f262800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.776+0000 7f2634ff9640 1 -- 172.21.15.49:0/4009796284 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26280052d0 con 0x7f26380b9f70 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.776+0000 7f2634ff9640 1 -- 172.21.15.49:0/4009796284 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628005430 con 0x7f26380b9f70 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.776+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26381e3be0 con 0x7f26380b9f70 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.776+0000 7f2634ff9640 1 -- 172.21.15.49:0/4009796284 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628003830 con 0x7f26380b9f70 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.776+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26380b3e70 con 0x7f26380b9f70 2024-03-14T21:14:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.777+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26381e3dd0 con 0x7f26380b9f70 2024-03-14T21:14:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.777+0000 7f2634ff9640 1 -- 172.21.15.49:0/4009796284 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2628003990 con 0x7f26380b9f70 2024-03-14T21:14:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.778+0000 7f2634ff9640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f262406a7a0 0x7f262406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.778+0000 7f2634ff9640 1 -- 172.21.15.49:0/4009796284 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2628088020 con 0x7f26380b9f70 2024-03-14T21:14:01.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.778+0000 7f26377fe640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f262406a7a0 0x7f262406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:01.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.779+0000 7f26377fe640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f262406a7a0 0x7f262406cc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f262c036000 tx=0x7f262c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.782+0000 7f2634ff9640 1 -- 172.21.15.49:0/4009796284 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2628053760 con 0x7f26380b9f70 2024-03-14T21:14:01.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:01.964+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f26380a6250 con 0x7f26380b9f70 2024-03-14T21:14:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.643+0000 7f2634ff9640 1 -- 172.21.15.49:0/4009796284 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2628056d80 con 0x7f26380b9f70 2024-03-14T21:14:02.627 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-03-14T21:14:02.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f262406a7a0 msgr2=0x7f262406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:02.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f262406a7a0 0x7f262406cc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f262c036000 tx=0x7f262c008030 comp rx=0 tx=0).stop 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380b9f70 msgr2=0x7f26381e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380b9f70 0x7f26381e43e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f262800c400 tx=0x7f262800c8c0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 shutdown_connections 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f262406a7a0 0x7f262406cc60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26380bfe50 0x7f26380b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26380bf9c0 0x7f26380a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 --2- 172.21.15.49:0/4009796284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26380b9f70 0x7f26381e43e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.653+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 >> 172.21.15.49:0/4009796284 conn(0x7f26380c02d0 msgr2=0x7f26380a1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:02.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.654+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 shutdown_connections 2024-03-14T21:14:02.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.654+0000 7f263e48b640 1 -- 172.21.15.49:0/4009796284 wait complete. 2024-03-14T21:14:02.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T21:14:02.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.781+0000 7f11694ce640 1 Processor -- start 2024-03-14T21:14:02.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11694ce640 1 -- start start 2024-03-14T21:14:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11694ce640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11640bad60 0x7f11640bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11694ce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11640b3a20 con 0x7f1164153590 2024-03-14T21:14:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11694ce640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116406adf0 con 0x7f116414b820 2024-03-14T21:14:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11694ce640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640ba550 con 0x7f11640bad60 2024-03-14T21:14:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11637fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1164153590 0x7f1164155980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36794/0 (socket says 172.21.15.49:36794) 2024-03-14T21:14:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11637fe640 1 -- 172.21.15.49:0/1399839643 learned_addr learned my addr 172.21.15.49:0/1399839643 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.782+0000 7f11627fc640 1 --2- 172.21.15.49:0/1399839643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11640bad60 0x7f11640bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f1161ffb640 1 -- 172.21.15.49:0/1399839643 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160907932 0 0) 0x7f11640b3a20 con 0x7f1164153590 2024-03-14T21:14:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f1161ffb640 1 -- 172.21.15.49:0/1399839643 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f114c003520 con 0x7f1164153590 2024-03-14T21:14:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f1161ffb640 1 -- 172.21.15.49:0/1399839643 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525953109 0 0) 0x7f116406adf0 con 0x7f116414b820 2024-03-14T21:14:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f1161ffb640 1 -- 172.21.15.49:0/1399839643 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11640b3a20 con 0x7f116414b820 2024-03-14T21:14:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f11627fc640 1 -- 172.21.15.49:0/1399839643 >> v1:172.21.15.97:6789/0 conn(0x7f116414b820 legacy=0x7f116406a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f11627fc640 1 -- 172.21.15.49:0/1399839643 >> v1:172.21.15.49:6789/0 conn(0x7f1164153590 legacy=0x7f1164155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f11627fc640 1 -- 172.21.15.49:0/1399839643 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640bfbe0 con 0x7f11640bad60 2024-03-14T21:14:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.783+0000 7f11627fc640 1 --2- 172.21.15.49:0/1399839643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11640bad60 0x7f11640bb1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11540097d0 tx=0x7f115402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f1ba1646193d895 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f1161ffb640 1 -- 172.21.15.49:0/1399839643 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154031890 con 0x7f11640bad60 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f1161ffb640 1 -- 172.21.15.49:0/1399839643 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11540319f0 con 0x7f11640bad60 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f1161ffb640 1 -- 172.21.15.49:0/1399839643 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154031d80 con 0x7f11640bad60 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f11694ce640 1 -- 172.21.15.49:0/1399839643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11640bad60 msgr2=0x7f11640bb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f11694ce640 1 --2- 172.21.15.49:0/1399839643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11640bad60 0x7f11640bb1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f11540097d0 tx=0x7f115402b930 comp rx=0 tx=0).stop 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f11694ce640 1 -- 172.21.15.49:0/1399839643 shutdown_connections 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f11694ce640 1 --2- 172.21.15.49:0/1399839643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f11640bad60 0x7f11640bb1c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f11694ce640 1 -- 172.21.15.49:0/1399839643 >> 172.21.15.49:0/1399839643 conn(0x7f11640bcd80 msgr2=0x7f11640bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f11694ce640 1 -- 172.21.15.49:0/1399839643 shutdown_connections 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.784+0000 7f11694ce640 1 -- 172.21.15.49:0/1399839643 wait complete. 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.785+0000 7f11694ce640 1 Processor -- start 2024-03-14T21:14:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11694ce640 1 -- start start 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11694ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11640bad60 0x7f11640b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11694ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f116414b820 0x7f116414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11694ce640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1164153590 0x7f1164144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11694ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640b6e60 con 0x7f116414b820 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11694ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116414ab60 con 0x7f11640bad60 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11694ce640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11641dfe30 con 0x7f1164153590 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11627fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f116414b820 0x7f116414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11627fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f116414b820 0x7f116414a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37272/0 (socket says 172.21.15.49:37272) 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f11627fc640 1 -- 172.21.15.49:0/200735681 learned_addr learned my addr 172.21.15.49:0/200735681 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.786+0000 7f1162ffd640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11640bad60 0x7f11640b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11637fe640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1164153590 0x7f1164144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11627fc640 1 -- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1164153590 msgr2=0x7f1164144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11627fc640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1164153590 0x7f1164144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11627fc640 1 -- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11640bad60 msgr2=0x7f11640b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11627fc640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11640bad60 0x7f11640b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11627fc640 1 -- 172.21.15.49:0/200735681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154009400 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11627fc640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f116414b820 0x7f116414a620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11540097d0 tx=0x7f11540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f114b7fe640 1 -- 172.21.15.49:0/200735681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115403abe0 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f114b7fe640 1 -- 172.21.15.49:0/200735681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115403ad60 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f114b7fe640 1 -- 172.21.15.49:0/200735681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154044630 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11640a58c0 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.787+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640a5e80 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.788+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11641dffe0 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.788+0000 7f114b7fe640 1 -- 172.21.15.49:0/200735681 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f115404d450 con 0x7f116414b820 2024-03-14T21:14:02.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.789+0000 7f114b7fe640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f115006a7d0 0x7f115006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:02.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.789+0000 7f114b7fe640 1 -- 172.21.15.49:0/200735681 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f11540ab2b0 con 0x7f116414b820 2024-03-14T21:14:02.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.789+0000 7f1162ffd640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f115006a7d0 0x7f115006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:02.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.790+0000 7f1162ffd640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f115006a7d0 0x7f115006cc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f11580011e0 tx=0x7f1158009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:02.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:02.794+0000 7f114b7fe640 1 -- 172.21.15.49:0/200735681 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11641dffe0 con 0x7f116414b820 2024-03-14T21:14:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.011+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f116404e150 con 0x7f116414b820 2024-03-14T21:14:03.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.019+0000 7f114b7fe640 1 -- 172.21.15.49:0/200735681 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f11540350c0 con 0x7f116414b820 2024-03-14T21:14:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f115006a7d0 msgr2=0x7f115006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f115006a7d0 0x7f115006cc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f11580011e0 tx=0x7f1158009040 comp rx=0 tx=0).stop 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f116414b820 msgr2=0x7f116414a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f116414b820 0x7f116414a620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11540097d0 tx=0x7f11540046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 shutdown_connections 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f115006a7d0 0x7f115006cc90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1164153590 0x7f1164144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f116414b820 0x7f116414a620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 --2- 172.21.15.49:0/200735681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11640bad60 0x7f11640b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 >> 172.21.15.49:0/200735681 conn(0x7f11640bcd80 msgr2=0x7f11640b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:03.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.026+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 shutdown_connections 2024-03-14T21:14:03.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.027+0000 7f11694ce640 1 -- 172.21.15.49:0/200735681 wait complete. 2024-03-14T21:14:03.028 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:03.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.150+0000 7f6db47d9640 1 Processor -- start 2024-03-14T21:14:03.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.151+0000 7f6db47d9640 1 -- start start 2024-03-14T21:14:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.151+0000 7f6db47d9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6dac06b620 0x7f6dac0bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.151+0000 7f6db47d9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6dac0bc940 0x7f6dac0bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.151+0000 7f6db47d9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dac06a6e0 con 0x7f6dac0baef0 2024-03-14T21:14:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.151+0000 7f6db47d9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dac06ad10 con 0x7f6dac0bc940 2024-03-14T21:14:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.151+0000 7f6db47d9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dac0bc2b0 con 0x7f6dac06b620 2024-03-14T21:14:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db2d76640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6dac0bc940 0x7f6dac0bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db2d76640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6dac0bc940 0x7f6dac0bfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37280/0 (socket says 172.21.15.49:37280) 2024-03-14T21:14:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db2d76640 1 -- 172.21.15.49:0/2317154181 learned_addr learned my addr 172.21.15.49:0/2317154181 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db1d74640 1 --2- 172.21.15.49:0/2317154181 >> v2:172.21.15.97:3300/0 conn(0x7f6dac06b620 0x7f6dac0bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db2d76640 1 -- 172.21.15.49:0/2317154181 >> v2:172.21.15.97:3300/0 conn(0x7f6dac06b620 msgr2=0x7f6dac0bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db2d76640 1 --2- 172.21.15.49:0/2317154181 >> v2:172.21.15.97:3300/0 conn(0x7f6dac06b620 0x7f6dac0bf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db2d76640 1 -- 172.21.15.49:0/2317154181 >> v1:172.21.15.97:6789/0 conn(0x7f6dac0baef0 legacy=0x7f6dac1480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.152+0000 7f6db2d76640 1 -- 172.21.15.49:0/2317154181 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac06bab0 con 0x7f6dac0bc940 2024-03-14T21:14:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db2d76640 1 --2- 172.21.15.49:0/2317154181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0bfd30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6dac04ea40 tx=0x7f6da402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60b53e8ab39e866 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db1573640 1 -- 172.21.15.49:0/2317154181 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da402ca80 con 0x7f6dac0bc940 2024-03-14T21:14:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db1573640 1 -- 172.21.15.49:0/2317154181 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da402cbe0 con 0x7f6dac0bc940 2024-03-14T21:14:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db1573640 1 -- 172.21.15.49:0/2317154181 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da40343f0 con 0x7f6dac0bc940 2024-03-14T21:14:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db47d9640 1 -- 172.21.15.49:0/2317154181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 msgr2=0x7f6dac0bfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2317154181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0bfd30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6dac04ea40 tx=0x7f6da402bee0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db47d9640 1 -- 172.21.15.49:0/2317154181 shutdown_connections 2024-03-14T21:14:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2317154181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0bfd30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2317154181 >> v2:172.21.15.97:3300/0 conn(0x7f6dac06b620 0x7f6dac0bf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db47d9640 1 -- 172.21.15.49:0/2317154181 >> 172.21.15.49:0/2317154181 conn(0x7f6dac0bcd80 msgr2=0x7f6dac0bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.153+0000 7f6db47d9640 1 -- 172.21.15.49:0/2317154181 shutdown_connections 2024-03-14T21:14:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.154+0000 7f6db47d9640 1 -- 172.21.15.49:0/2317154181 wait complete. 2024-03-14T21:14:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 Processor -- start 2024-03-14T21:14:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 -- start start 2024-03-14T21:14:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6dac0baef0 0x7f6dac0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac0b6df0 0x7f6dac1dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac1d5c10 con 0x7f6dac0bc940 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac0b4090 con 0x7f6dac0b6df0 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db47d9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac0a1ce0 con 0x7f6dac0baef0 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db1d74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db1d74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37282/0 (socket says 172.21.15.49:37282) 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db1d74640 1 -- 172.21.15.49:0/2145041315 learned_addr learned my addr 172.21.15.49:0/2145041315 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db2575640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6dac0baef0 0x7f6dac0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.155+0000 7f6db2d76640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac0b6df0 0x7f6dac1dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db1d74640 1 -- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6dac0baef0 msgr2=0x7f6dac0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db1d74640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6dac0baef0 0x7f6dac0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db1d74640 1 -- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac0b6df0 msgr2=0x7f6dac1dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db1d74640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac0b6df0 0x7f6dac1dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db1d74640 1 -- 172.21.15.49:0/2145041315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da4009b30 con 0x7f6dac0bc940 2024-03-14T21:14:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db1d74640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0b68b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e9b0 tx=0x7f6da800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6d9effd640 1 -- 172.21.15.49:0/2145041315 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8009670 con 0x7f6dac0bc940 2024-03-14T21:14:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6d9effd640 1 -- 172.21.15.49:0/2145041315 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800c9a0 con 0x7f6dac0bc940 2024-03-14T21:14:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dac148940 con 0x7f6dac0bc940 2024-03-14T21:14:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6d9effd640 1 -- 172.21.15.49:0/2145041315 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800cd50 con 0x7f6dac0bc940 2024-03-14T21:14:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.156+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dac0a22a0 con 0x7f6dac0bc940 2024-03-14T21:14:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.157+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dac066bb0 con 0x7f6dac0bc940 2024-03-14T21:14:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.157+0000 7f6d9effd640 1 -- 172.21.15.49:0/2145041315 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6da8013470 con 0x7f6dac0bc940 2024-03-14T21:14:03.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.158+0000 7f6d9effd640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6d9006a7d0 0x7f6d9006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.158+0000 7f6d9effd640 1 -- 172.21.15.49:0/2145041315 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6da8088f30 con 0x7f6dac0bc940 2024-03-14T21:14:03.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.158+0000 7f6db2575640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6d9006a7d0 0x7f6d9006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.159+0000 7f6db2575640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6d9006a7d0 0x7f6d9006cc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6da0000f60 tx=0x7f6da0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.162+0000 7f6d9effd640 1 -- 172.21.15.49:0/2145041315 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6da8054400 con 0x7f6dac0bc940 2024-03-14T21:14:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.342+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dac146de0 con 0x7f6dac0bc940 2024-03-14T21:14:03.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.343+0000 7f6d9effd640 1 -- 172.21.15.49:0/2145041315 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16002 (secure 0 0 0) 0x7f6da8057a20 con 0x7f6dac0bc940 2024-03-14T21:14:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:03.326 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:03.012447+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6d9006a7d0 msgr2=0x7f6d9006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6d9006a7d0 0x7f6d9006cc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6da0000f60 tx=0x7f6da0009040 comp rx=0 tx=0).stop 2024-03-14T21:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 msgr2=0x7f6dac0b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0b68b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e9b0 tx=0x7f6da800ee70 comp rx=0 tx=0).stop 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 shutdown_connections 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6d9006a7d0 0x7f6d9006cc90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac0b6df0 0x7f6dac1dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6dac0bc940 0x7f6dac0b68b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 --2- 172.21.15.49:0/2145041315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6dac0baef0 0x7f6dac0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 >> 172.21.15.49:0/2145041315 conn(0x7f6dac0bcd80 msgr2=0x7f6dac14a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 shutdown_connections 2024-03-14T21:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.345+0000 7f6db47d9640 1 -- 172.21.15.49:0/2145041315 wait complete. 2024-03-14T21:14:03.339 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.461+0000 7f6137bb9640 1 Processor -- start 2024-03-14T21:14:03.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.461+0000 7f6137bb9640 1 -- start start 2024-03-14T21:14:03.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.461+0000 7f6137bb9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f613006b620 0x7f61300bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.461+0000 7f6137bb9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61300bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.461+0000 7f6137bb9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613006a6e0 con 0x7f61300baef0 2024-03-14T21:14:03.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.461+0000 7f6137bb9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613006ad10 con 0x7f613006b620 2024-03-14T21:14:03.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.461+0000 7f6137bb9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61300bc2b0 con 0x7f61300bc940 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6135154640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f613006b620 0x7f61300bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6136156640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61300bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6135955640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f61300baef0 0x7f61301480b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47626/0 (socket says 172.21.15.49:47626) 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6135955640 1 -- 172.21.15.49:0/3200092735 learned_addr learned my addr 172.21.15.49:0/3200092735 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6136156640 1 -- 172.21.15.49:0/3200092735 >> v2:172.21.15.97:3300/0 conn(0x7f613006b620 msgr2=0x7f61300bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6136156640 1 --2- 172.21.15.49:0/3200092735 >> v2:172.21.15.97:3300/0 conn(0x7f613006b620 0x7f61300bf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6136156640 1 -- 172.21.15.49:0/3200092735 >> v1:172.21.15.97:6789/0 conn(0x7f61300baef0 legacy=0x7f61301480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.462+0000 7f6136156640 1 -- 172.21.15.49:0/3200092735 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613006bab0 con 0x7f61300bc940 2024-03-14T21:14:03.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.463+0000 7f6136156640 1 --2- 172.21.15.49:0/3200092735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61300bfd30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6128009d90 tx=0x7f612802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83fda4b6729d5330 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.463+0000 7f6134953640 1 -- 172.21.15.49:0/3200092735 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612802cbc0 con 0x7f61300bc940 2024-03-14T21:14:03.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.463+0000 7f6134953640 1 -- 172.21.15.49:0/3200092735 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612802cd20 con 0x7f61300bc940 2024-03-14T21:14:03.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.463+0000 7f6134953640 1 -- 172.21.15.49:0/3200092735 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128034660 con 0x7f61300bc940 2024-03-14T21:14:03.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 -- 172.21.15.49:0/3200092735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 msgr2=0x7f61300bfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3200092735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61300bfd30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6128009d90 tx=0x7f612802be10 comp rx=0 tx=0).stop 2024-03-14T21:14:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 -- 172.21.15.49:0/3200092735 shutdown_connections 2024-03-14T21:14:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3200092735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61300bfd30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3200092735 >> v2:172.21.15.97:3300/0 conn(0x7f613006b620 0x7f61300bf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 -- 172.21.15.49:0/3200092735 >> 172.21.15.49:0/3200092735 conn(0x7f61300bcd80 msgr2=0x7f61300bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 -- 172.21.15.49:0/3200092735 shutdown_connections 2024-03-14T21:14:03.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.464+0000 7f6137bb9640 1 -- 172.21.15.49:0/3200092735 wait complete. 2024-03-14T21:14:03.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 Processor -- start 2024-03-14T21:14:03.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 -- start start 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61300baef0 0x7f6130146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61301e1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f613014e610 0x7f61301e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61300b46e0 con 0x7f613014e610 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6130146a90 con 0x7f61300baef0 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6137bb9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613014f340 con 0x7f61300bc940 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.465+0000 7f6136156640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f613014e610 0x7f61301e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f613014e610 0x7f61301e16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37290/0 (socket says 172.21.15.49:37290) 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 -- 172.21.15.49:0/3218429424 learned_addr learned my addr 172.21.15.49:0/3218429424 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6135955640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61300baef0 0x7f6130146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6135154640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61301e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 -- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 msgr2=0x7f61301e1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61301e1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 -- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61300baef0 msgr2=0x7f6130146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61300baef0 0x7f6130146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 -- 172.21.15.49:0/3218429424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128009a70 con 0x7f613014e610 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6135154640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61301e1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6136156640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f613014e610 0x7f61301e16c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6128009eb0 tx=0x7f6128004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f61267fc640 1 -- 172.21.15.49:0/3218429424 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612803e610 con 0x7f613014e610 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f61267fc640 1 -- 172.21.15.49:0/3218429424 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612802ce90 con 0x7f613014e610 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6130148940 con 0x7f613014e610 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f61267fc640 1 -- 172.21.15.49:0/3218429424 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612803e9b0 con 0x7f613014e610 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.466+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613014eaa0 con 0x7f613014e610 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.467+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61301e0e50 con 0x7f613014e610 2024-03-14T21:14:03.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.468+0000 7f61267fc640 1 -- 172.21.15.49:0/3218429424 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f612803b050 con 0x7f613014e610 2024-03-14T21:14:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.468+0000 7f61267fc640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f611406a7a0 0x7f611406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.468+0000 7f61267fc640 1 -- 172.21.15.49:0/3218429424 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61280aaee0 con 0x7f613014e610 2024-03-14T21:14:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.468+0000 7f6135955640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f611406a7a0 0x7f611406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.469+0000 7f6135955640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f611406a7a0 0x7f611406cc60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f612c000f60 tx=0x7f612c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.472+0000 7f61267fc640 1 -- 172.21.15.49:0/3218429424 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6128076390 con 0x7f613014e610 2024-03-14T21:14:03.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.652+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61300666a0 con 0x7f613014e610 2024-03-14T21:14:03.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.653+0000 7f61267fc640 1 -- 172.21.15.49:0/3218429424 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16002 (secure 0 0 0) 0x7f612804d090 con 0x7f613014e610 2024-03-14T21:14:03.636 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:03.636 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:03.012447+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f611406a7a0 msgr2=0x7f611406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f611406a7a0 0x7f611406cc60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f612c000f60 tx=0x7f612c009040 comp rx=0 tx=0).stop 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f613014e610 msgr2=0x7f61301e16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f613014e610 0x7f61301e16c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6128009eb0 tx=0x7f6128004a40 comp rx=0 tx=0).stop 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 shutdown_connections 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f611406a7a0 0x7f611406cc60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f613014e610 0x7f61301e16c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61300bc940 0x7f61301e1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 --2- 172.21.15.49:0/3218429424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61300baef0 0x7f6130146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 >> 172.21.15.49:0/3218429424 conn(0x7f61300bcd80 msgr2=0x7f6130072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 shutdown_connections 2024-03-14T21:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.655+0000 7f6137bb9640 1 -- 172.21.15.49:0/3218429424 wait complete. 2024-03-14T21:14:03.649 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T21:14:03.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.774+0000 7efd28dfd640 1 Processor -- start 2024-03-14T21:14:03.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.774+0000 7efd28dfd640 1 -- start start 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd28dfd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efd24144fd0 0x7efd240b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd28dfd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efd2414e180 0x7efd2414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd28dfd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd24071e40 con 0x7efd240b7b30 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd28dfd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd24071040 con 0x7efd24144fd0 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd28dfd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2406a4d0 con 0x7efd2414e180 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd2259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efd24144fd0 0x7efd240b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd2259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efd24144fd0 0x7efd240b9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37300/0 (socket says 172.21.15.49:37300) 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd2259c640 1 -- 172.21.15.49:0/1928044114 learned_addr learned my addr 172.21.15.49:0/1928044114 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.775+0000 7efd21d9b640 1 --2- 172.21.15.49:0/1928044114 >> v2:172.21.15.97:3300/0 conn(0x7efd2414e180 0x7efd2414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2259c640 1 -- 172.21.15.49:0/1928044114 >> v2:172.21.15.97:3300/0 conn(0x7efd2414e180 msgr2=0x7efd2414e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:03.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2259c640 1 --2- 172.21.15.49:0/1928044114 >> v2:172.21.15.97:3300/0 conn(0x7efd2414e180 0x7efd2414e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2259c640 1 -- 172.21.15.49:0/1928044114 >> v1:172.21.15.97:6789/0 conn(0x7efd240b7b30 legacy=0x7efd240b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2259c640 1 -- 172.21.15.49:0/1928044114 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2414eb00 con 0x7efd24144fd0 2024-03-14T21:14:03.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2259c640 1 --2- 172.21.15.49:0/1928044114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd24144fd0 0x7efd240b9330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd2406f7b0 tx=0x7efd1c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3429e79e05a3867a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2159a640 1 -- 172.21.15.49:0/1928044114 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c02cb20 con 0x7efd24144fd0 2024-03-14T21:14:03.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2159a640 1 -- 172.21.15.49:0/1928044114 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c02cc80 con 0x7efd24144fd0 2024-03-14T21:14:03.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.776+0000 7efd2159a640 1 -- 172.21.15.49:0/1928044114 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c0343f0 con 0x7efd24144fd0 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 -- 172.21.15.49:0/1928044114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd24144fd0 msgr2=0x7efd240b9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 --2- 172.21.15.49:0/1928044114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd24144fd0 0x7efd240b9330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd2406f7b0 tx=0x7efd1c02bf80 comp rx=0 tx=0).stop 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 -- 172.21.15.49:0/1928044114 shutdown_connections 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 --2- 172.21.15.49:0/1928044114 >> v2:172.21.15.97:3300/0 conn(0x7efd2414e180 0x7efd2414e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 --2- 172.21.15.49:0/1928044114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd24144fd0 0x7efd240b9330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 -- 172.21.15.49:0/1928044114 >> 172.21.15.49:0/1928044114 conn(0x7efd240b9c30 msgr2=0x7efd240ba040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 -- 172.21.15.49:0/1928044114 shutdown_connections 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.777+0000 7efd28dfd640 1 -- 172.21.15.49:0/1928044114 wait complete. 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.778+0000 7efd28dfd640 1 Processor -- start 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.778+0000 7efd28dfd640 1 -- start start 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd28dfd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efd240b7b30 0x7efd2414ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd28dfd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd24144fd0 0x7efd24149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd28dfd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2414e180 0x7efd240a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd28dfd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2414b080 con 0x7efd2414e180 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd28dfd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2414a450 con 0x7efd24144fd0 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd28dfd640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd24148860 con 0x7efd240b7b30 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2414e180 0x7efd240a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2414e180 0x7efd240a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37308/0 (socket says 172.21.15.49:37308) 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 -- 172.21.15.49:0/2501301935 learned_addr learned my addr 172.21.15.49:0/2501301935 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd2259c640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efd240b7b30 0x7efd2414ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd21d9b640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd24144fd0 0x7efd24149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 -- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efd240b7b30 msgr2=0x7efd2414ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efd240b7b30 0x7efd2414ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 -- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd24144fd0 msgr2=0x7efd24149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd24144fd0 0x7efd24149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 -- 172.21.15.49:0/2501301935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c009b30 con 0x7efd2414e180 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.779+0000 7efd22d9d640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2414e180 0x7efd240a0350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efd1400c3c0 tx=0x7efd1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.780+0000 7efd12ffd640 1 -- 172.21.15.49:0/2501301935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd140052d0 con 0x7efd2414e180 2024-03-14T21:14:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.780+0000 7efd12ffd640 1 -- 172.21.15.49:0/2501301935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd14005430 con 0x7efd2414e180 2024-03-14T21:14:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.780+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd240a33b0 con 0x7efd2414e180 2024-03-14T21:14:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.780+0000 7efd12ffd640 1 -- 172.21.15.49:0/2501301935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14003810 con 0x7efd2414e180 2024-03-14T21:14:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.780+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd241499f0 con 0x7efd2414e180 2024-03-14T21:14:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.781+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd24149570 con 0x7efd2414e180 2024-03-14T21:14:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.781+0000 7efd12ffd640 1 -- 172.21.15.49:0/2501301935 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7efd14003970 con 0x7efd2414e180 2024-03-14T21:14:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.782+0000 7efd12ffd640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7efd0406a7a0 0x7efd0406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.782+0000 7efd12ffd640 1 -- 172.21.15.49:0/2501301935 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd14088120 con 0x7efd2414e180 2024-03-14T21:14:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.782+0000 7efd2259c640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7efd0406a7a0 0x7efd0406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.782+0000 7efd2259c640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7efd0406a7a0 0x7efd0406cc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efd1c035000 tx=0x7efd1c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:03.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:03.786+0000 7efd12ffd640 1 -- 172.21.15.49:0/2501301935 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efd24149570 con 0x7efd2414e180 2024-03-14T21:14:03.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.002+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efd24149570 con 0x7efd2414e180 2024-03-14T21:14:04.548 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:14:04.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:14:04.554 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:14:04.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.653+0000 7efd12ffd640 1 -- 172.21.15.49:0/2501301935 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7efd24149570 con 0x7efd2414e180 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.655+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7efd0406a7a0 msgr2=0x7efd0406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.655+0000 7efd28dfd640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7efd0406a7a0 0x7efd0406cc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efd1c035000 tx=0x7efd1c030000 comp rx=0 tx=0).stop 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.655+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2414e180 msgr2=0x7efd240a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.655+0000 7efd28dfd640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2414e180 0x7efd240a0350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efd1400c3c0 tx=0x7efd1400c880 comp rx=0 tx=0).stop 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 shutdown_connections 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7efd0406a7a0 0x7efd0406cc60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd2414e180 0x7efd240a0350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd24144fd0 0x7efd24149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 --2- 172.21.15.49:0/2501301935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efd240b7b30 0x7efd2414ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 >> 172.21.15.49:0/2501301935 conn(0x7efd240b9c30 msgr2=0x7efd240bb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:04.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 shutdown_connections 2024-03-14T21:14:04.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.656+0000 7efd28dfd640 1 -- 172.21.15.49:0/2501301935 wait complete. 2024-03-14T21:14:04.652 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T21:14:04.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T21:14:04.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.824+0000 7f00c8727640 1 Processor -- start 2024-03-14T21:14:04.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c8727640 1 -- start start 2024-03-14T21:14:04.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c8727640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f00c014b820 0x7f00c006a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:04.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c8727640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00c00bad60 0x7f00c00bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:04.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c8727640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c00b3a20 con 0x7f00c014e270 2024-03-14T21:14:04.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c8727640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c00ba550 con 0x7f00c014b820 2024-03-14T21:14:04.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c8727640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c00bfbe0 con 0x7f00c00bad60 2024-03-14T21:14:04.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c64c3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f00c014b820 0x7f00c006a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:04.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c64c3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f00c014b820 0x7f00c006a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37324/0 (socket says 172.21.15.49:37324) 2024-03-14T21:14:04.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.825+0000 7f00c64c3640 1 -- 172.21.15.49:0/3370456609 learned_addr learned my addr 172.21.15.49:0/3370456609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:04.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.826+0000 7f00c5cc2640 1 --2- 172.21.15.49:0/3370456609 >> v2:172.21.15.97:3300/0 conn(0x7f00c00bad60 0x7f00c00bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:04.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.826+0000 7f00c64c3640 1 -- 172.21.15.49:0/3370456609 >> v2:172.21.15.97:3300/0 conn(0x7f00c00bad60 msgr2=0x7f00c00bb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:04.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.826+0000 7f00c64c3640 1 --2- 172.21.15.49:0/3370456609 >> v2:172.21.15.97:3300/0 conn(0x7f00c00bad60 0x7f00c00bb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.826+0000 7f00c64c3640 1 -- 172.21.15.49:0/3370456609 >> v1:172.21.15.97:6789/0 conn(0x7f00c014e270 legacy=0x7f00c014e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:04.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.826+0000 7f00c64c3640 1 -- 172.21.15.49:0/3370456609 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c00bfec0 con 0x7f00c014b820 2024-03-14T21:14:04.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.826+0000 7f00c5cc2640 1 --2- 172.21.15.49:0/3370456609 >> v2:172.21.15.97:3300/0 conn(0x7f00c00bad60 0x7f00c00bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:04.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c64c3640 1 --2- 172.21.15.49:0/3370456609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c006a6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00c0061a20 tx=0x7f00bc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db22dac62e12b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:04.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c54c1640 1 -- 172.21.15.49:0/3370456609 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc02cb20 con 0x7f00c014b820 2024-03-14T21:14:04.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c54c1640 1 -- 172.21.15.49:0/3370456609 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc02cc80 con 0x7f00c014b820 2024-03-14T21:14:04.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c54c1640 1 -- 172.21.15.49:0/3370456609 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0343f0 con 0x7f00c014b820 2024-03-14T21:14:04.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 -- 172.21.15.49:0/3370456609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 msgr2=0x7f00c006a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:04.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 --2- 172.21.15.49:0/3370456609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c006a6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00c0061a20 tx=0x7f00bc02bf80 comp rx=0 tx=0).stop 2024-03-14T21:14:04.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 -- 172.21.15.49:0/3370456609 shutdown_connections 2024-03-14T21:14:04.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 --2- 172.21.15.49:0/3370456609 >> v2:172.21.15.97:3300/0 conn(0x7f00c00bad60 0x7f00c00bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 --2- 172.21.15.49:0/3370456609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c006a6e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 -- 172.21.15.49:0/3370456609 >> 172.21.15.49:0/3370456609 conn(0x7f00c00bcd80 msgr2=0x7f00c00bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:04.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 -- 172.21.15.49:0/3370456609 shutdown_connections 2024-03-14T21:14:04.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.827+0000 7f00c8727640 1 -- 172.21.15.49:0/3370456609 wait complete. 2024-03-14T21:14:04.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.828+0000 7f00c8727640 1 Processor -- start 2024-03-14T21:14:04.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.828+0000 7f00c8727640 1 -- start start 2024-03-14T21:14:04.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c8727640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c00bad60 0x7f00c01d5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:04.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c8727640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c01e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:04.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c8727640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00c014e270 0x7f00c01d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:04.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c8727640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01dfb10 con 0x7f00c014b820 2024-03-14T21:14:04.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c8727640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01d5100 con 0x7f00c00bad60 2024-03-14T21:14:04.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c8727640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01e0030 con 0x7f00c014e270 2024-03-14T21:14:04.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c5cc2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c01e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:04.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c5cc2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c01e04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37334/0 (socket says 172.21.15.49:37334) 2024-03-14T21:14:04.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c5cc2640 1 -- 172.21.15.49:0/1548353249 learned_addr learned my addr 172.21.15.49:0/1548353249 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:04.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c64c3640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c00bad60 0x7f00c01d5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:04.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c6cc4640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00c014e270 0x7f00c01d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:04.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c5cc2640 1 -- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00c014e270 msgr2=0x7f00c01d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:04.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.829+0000 7f00c5cc2640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00c014e270 0x7f00c01d52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00c5cc2640 1 -- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c00bad60 msgr2=0x7f00c01d5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00c5cc2640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c00bad60 0x7f00c01d5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00c5cc2640 1 -- 172.21.15.49:0/1548353249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc009b30 con 0x7f00c014b820 2024-03-14T21:14:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00c5cc2640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c01e04c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b000d800 tx=0x7f00b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00b6ffd640 1 -- 172.21.15.49:0/1548353249 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00096a0 con 0x7f00c014b820 2024-03-14T21:14:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00b6ffd640 1 -- 172.21.15.49:0/1548353249 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b0009800 con 0x7f00c014b820 2024-03-14T21:14:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00b6ffd640 1 -- 172.21.15.49:0/1548353249 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b0009bb0 con 0x7f00c014b820 2024-03-14T21:14:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c01dfd00 con 0x7f00c014b820 2024-03-14T21:14:04.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.830+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c00b4050 con 0x7f00c014b820 2024-03-14T21:14:04.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.831+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c00bb4f0 con 0x7f00c014b820 2024-03-14T21:14:04.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.831+0000 7f00b6ffd640 1 -- 172.21.15.49:0/1548353249 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f00b0009d10 con 0x7f00c014b820 2024-03-14T21:14:04.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.832+0000 7f00b6ffd640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f00a406a7a0 0x7f00a406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:04.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.832+0000 7f00b6ffd640 1 -- 172.21.15.49:0/1548353249 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00b0089050 con 0x7f00c014b820 2024-03-14T21:14:04.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.832+0000 7f00c64c3640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f00a406a7a0 0x7f00a406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:04.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.832+0000 7f00c64c3640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f00a406a7a0 0x7f00a406cc60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00bc035000 tx=0x7f00bc030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:04.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:04.836+0000 7f00b6ffd640 1 -- 172.21.15.49:0/1548353249 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f00b00545b0 con 0x7f00c014b820 2024-03-14T21:14:05.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.056+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00c0066bb0 con 0x7f00c014b820 2024-03-14T21:14:05.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.661+0000 7f00b6ffd640 1 -- 172.21.15.49:0/1548353249 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f00b0057bd0 con 0x7f00c014b820 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f00a406a7a0 msgr2=0x7f00a406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f00a406a7a0 0x7f00a406cc60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00bc035000 tx=0x7f00bc030000 comp rx=0 tx=0).stop 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 msgr2=0x7f00c01e04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c01e04c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b000d800 tx=0x7f00b000dcc0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 shutdown_connections 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f00a406a7a0 0x7f00a406cc60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00c014e270 0x7f00c01d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00c014b820 0x7f00c01e04c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 --2- 172.21.15.49:0/1548353249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c00bad60 0x7f00c01d5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 >> 172.21.15.49:0/1548353249 conn(0x7f00c00bcd80 msgr2=0x7f00c014a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:05.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 shutdown_connections 2024-03-14T21:14:05.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.664+0000 7f00c8727640 1 -- 172.21.15.49:0/1548353249 wait complete. 2024-03-14T21:14:05.660 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T21:14:05.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T21:14:05.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.788+0000 7f9b7effb640 1 Processor -- start 2024-03-14T21:14:05.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.788+0000 7f9b7effb640 1 -- start start 2024-03-14T21:14:05.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b7effb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9b7814e180 0x7f9b7814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:05.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b7effb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b781533a0 0x7f9b78155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:05.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b7effb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b780b9a40 con 0x7f9b78144fd0 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b7effb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b78071040 con 0x7f9b7814e180 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b7effb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b7806a4d0 con 0x7f9b781533a0 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b77fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9b7814e180 0x7f9b7814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b77fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9b7814e180 0x7f9b7814e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37336/0 (socket says 172.21.15.49:37336) 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b77fff640 1 -- 172.21.15.49:0/2087896240 learned_addr learned my addr 172.21.15.49:0/2087896240 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b7cd97640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9b78144fd0 0x7f9b780b9330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36796/0 (socket says 172.21.15.49:36796) 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b7d598640 1 --2- 172.21.15.49:0/2087896240 >> v2:172.21.15.97:3300/0 conn(0x7f9b781533a0 0x7f9b78155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b77fff640 1 -- 172.21.15.49:0/2087896240 >> v2:172.21.15.97:3300/0 conn(0x7f9b781533a0 msgr2=0x7f9b78155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b77fff640 1 --2- 172.21.15.49:0/2087896240 >> v2:172.21.15.97:3300/0 conn(0x7f9b781533a0 0x7f9b78155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b77fff640 1 -- 172.21.15.49:0/2087896240 >> v1:172.21.15.49:6789/0 conn(0x7f9b78144fd0 legacy=0x7f9b780b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.789+0000 7f9b77fff640 1 -- 172.21.15.49:0/2087896240 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78071e40 con 0x7f9b7814e180 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.790+0000 7f9b77fff640 1 --2- 172.21.15.49:0/2087896240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7814e5c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b7806c690 tx=0x7f9b6c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c30222ea7a0242 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.790+0000 7f9b777fe640 1 -- 172.21.15.49:0/2087896240 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c030860 con 0x7f9b7814e180 2024-03-14T21:14:05.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.790+0000 7f9b777fe640 1 -- 172.21.15.49:0/2087896240 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0309c0 con 0x7f9b7814e180 2024-03-14T21:14:05.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.790+0000 7f9b777fe640 1 -- 172.21.15.49:0/2087896240 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c030d50 con 0x7f9b7814e180 2024-03-14T21:14:05.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 -- 172.21.15.49:0/2087896240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 msgr2=0x7f9b7814e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 --2- 172.21.15.49:0/2087896240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7814e5c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b7806c690 tx=0x7f9b6c02b9a0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 -- 172.21.15.49:0/2087896240 shutdown_connections 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 --2- 172.21.15.49:0/2087896240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7814e5c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 --2- 172.21.15.49:0/2087896240 >> v2:172.21.15.97:3300/0 conn(0x7f9b781533a0 0x7f9b78155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 -- 172.21.15.49:0/2087896240 >> 172.21.15.49:0/2087896240 conn(0x7f9b780b9c30 msgr2=0x7f9b780ba040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 -- 172.21.15.49:0/2087896240 shutdown_connections 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.791+0000 7f9b7effb640 1 -- 172.21.15.49:0/2087896240 wait complete. 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.792+0000 7f9b7effb640 1 Processor -- start 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.792+0000 7f9b7effb640 1 -- start start 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7effb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b78144fd0 0x7f9b7813ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7effb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7809ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7effb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b781533a0 0x7f9b7809d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7effb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7813cf80 con 0x7f9b7814e180 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7effb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7809ce60 con 0x7f9b781533a0 2024-03-14T21:14:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7effb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b78147fd0 con 0x7f9b78144fd0 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7809ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7809ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37346/0 (socket says 172.21.15.49:37346) 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 -- 172.21.15.49:0/3342609661 learned_addr learned my addr 172.21.15.49:0/3342609661 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7cd97640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b78144fd0 0x7f9b7813ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b7d598640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b781533a0 0x7f9b7809d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 -- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b78144fd0 msgr2=0x7f9b7813ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b78144fd0 0x7f9b7813ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 -- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b781533a0 msgr2=0x7f9b7809d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b781533a0 0x7f9b7809d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 -- 172.21.15.49:0/3342609661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c0094d0 con 0x7f9b7814e180 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b77fff640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7809ebe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c02b9e0 tx=0x7f9b6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:05.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.793+0000 7f9b757fa640 1 -- 172.21.15.49:0/3342609661 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c039c90 con 0x7f9b7814e180 2024-03-14T21:14:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.794+0000 7f9b757fa640 1 -- 172.21.15.49:0/3342609661 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c039e10 con 0x7f9b7814e180 2024-03-14T21:14:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.794+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b780a3380 con 0x7f9b7814e180 2024-03-14T21:14:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.794+0000 7f9b757fa640 1 -- 172.21.15.49:0/3342609661 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c043640 con 0x7f9b7814e180 2024-03-14T21:14:05.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.794+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7809f120 con 0x7f9b7814e180 2024-03-14T21:14:05.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.795+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7806a0b0 con 0x7f9b7814e180 2024-03-14T21:14:05.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.795+0000 7f9b757fa640 1 -- 172.21.15.49:0/3342609661 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9b6c0437a0 con 0x7f9b7814e180 2024-03-14T21:14:05.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.795+0000 7f9b757fa640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9b6406a7a0 0x7f9b6406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:05.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.796+0000 7f9b757fa640 1 -- 172.21.15.49:0/3342609661 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b6c03e070 con 0x7f9b7814e180 2024-03-14T21:14:05.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.796+0000 7f9b7cd97640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9b6406a7a0 0x7f9b6406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:05.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.796+0000 7f9b7cd97640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9b6406a7a0 0x7f9b6406cc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b700012e0 tx=0x7f9b70001650 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:05.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:05.800+0000 7f9b757fa640 1 -- 172.21.15.49:0/3342609661 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b6c077290 con 0x7f9b7814e180 2024-03-14T21:14:05.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.014+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b78073540 con 0x7f9b7814e180 2024-03-14T21:14:05.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.014+0000 7f9b757fa640 1 -- 172.21.15.49:0/3342609661 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3969 (secure 0 0 0) 0x7f9b6c034070 con 0x7f9b7814e180 2024-03-14T21:14:05.997 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:05.998 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2343109927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2343109927},{"type":"v1","addr":"172.21.15.97:6835","nonce":2343109927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1491968682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1491968682},{"type":"v1","addr":"172.21.15.49:6833","nonce":1491968682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T21:14:03.012761+0000","modified":"2024-03-14T21:14:05.655989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T21:14:05.998 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-14T21:14:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9b6406a7a0 msgr2=0x7f9b6406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9b6406a7a0 0x7f9b6406cc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b700012e0 tx=0x7f9b70001650 comp rx=0 tx=0).stop 2024-03-14T21:14:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 msgr2=0x7f9b7809ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7809ebe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c02b9e0 tx=0x7f9b6c004730 comp rx=0 tx=0).stop 2024-03-14T21:14:05.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 shutdown_connections 2024-03-14T21:14:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9b6406a7a0 0x7f9b6406cc60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b781533a0 0x7f9b7809d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b7814e180 0x7f9b7809ebe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 --2- 172.21.15.49:0/3342609661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b78144fd0 0x7f9b7813ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 >> 172.21.15.49:0/3342609661 conn(0x7f9b780b9c30 msgr2=0x7f9b780b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 shutdown_connections 2024-03-14T21:14:06.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.016+0000 7f9b7effb640 1 -- 172.21.15.49:0/3342609661 wait complete. 2024-03-14T21:14:06.010 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.132+0000 7fa96ff51640 1 Processor -- start 2024-03-14T21:14:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.132+0000 7fa96ff51640 1 -- start start 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96ff51640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9680baef0 0x7fa9681480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96ff51640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96806a6e0 con 0x7fa9680bc940 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96ff51640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9680bc2b0 con 0x7fa96806b620 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96ff51640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96806ad10 con 0x7fa9680baef0 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96e4ee640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa9680bc940 0x7fa968155710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36812/0 (socket says 172.21.15.49:36812) 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96e4ee640 1 -- 172.21.15.49:0/150638135 learned_addr learned my addr 172.21.15.49:0/150638135 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96dced640 1 --2- 172.21.15.49:0/150638135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9680baef0 0x7fa9681480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96cceb640 1 -- 172.21.15.49:0/150638135 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666385694 0 0) 0x7fa96806a6e0 con 0x7fa9680bc940 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96cceb640 1 -- 172.21.15.49:0/150638135 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa954003520 con 0x7fa9680bc940 2024-03-14T21:14:06.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96cceb640 1 -- 172.21.15.49:0/150638135 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374640951 0 0) 0x7fa9680bc2b0 con 0x7fa96806b620 2024-03-14T21:14:06.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96cceb640 1 -- 172.21.15.49:0/150638135 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa96806a6e0 con 0x7fa96806b620 2024-03-14T21:14:06.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96dced640 1 -- 172.21.15.49:0/150638135 >> v1:172.21.15.97:6789/0 conn(0x7fa96806b620 legacy=0x7fa9680bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96dced640 1 -- 172.21.15.49:0/150638135 >> v1:172.21.15.49:6789/0 conn(0x7fa9680bc940 legacy=0x7fa968155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.133+0000 7fa96dced640 1 -- 172.21.15.49:0/150638135 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96806bab0 con 0x7fa9680baef0 2024-03-14T21:14:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.134+0000 7fa96dced640 1 --2- 172.21.15.49:0/150638135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9680baef0 0x7fa9681480b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097d0 tx=0x7fa96402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d6152564b49915 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.134+0000 7fa96cceb640 1 -- 172.21.15.49:0/150638135 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964031800 con 0x7fa9680baef0 2024-03-14T21:14:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96cceb640 1 -- 172.21.15.49:0/150638135 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964031960 con 0x7fa9680baef0 2024-03-14T21:14:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96cceb640 1 -- 172.21.15.49:0/150638135 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964031cd0 con 0x7fa9680baef0 2024-03-14T21:14:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96ff51640 1 -- 172.21.15.49:0/150638135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9680baef0 msgr2=0x7fa9681480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96ff51640 1 --2- 172.21.15.49:0/150638135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9680baef0 0x7fa9681480b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097d0 tx=0x7fa96402b8a0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96ff51640 1 -- 172.21.15.49:0/150638135 shutdown_connections 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96ff51640 1 --2- 172.21.15.49:0/150638135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9680baef0 0x7fa9681480b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96ff51640 1 -- 172.21.15.49:0/150638135 >> 172.21.15.49:0/150638135 conn(0x7fa9680bcd80 msgr2=0x7fa9680bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96ff51640 1 -- 172.21.15.49:0/150638135 shutdown_connections 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.135+0000 7fa96ff51640 1 -- 172.21.15.49:0/150638135 wait complete. 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.136+0000 7fa96ff51640 1 Processor -- start 2024-03-14T21:14:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.136+0000 7fa96ff51640 1 -- start start 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96ff51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9680baef0 0x7fa9681dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96ff51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9680bc940 0x7fa9680b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96ff51640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa96814e3b0 0x7fa9681e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96ff51640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681e03a0 con 0x7fa9680bc940 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96ff51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9680b2b10 con 0x7fa9680baef0 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96ff51640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9680b69c0 con 0x7fa96814e3b0 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9680bc940 0x7fa9680b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9680bc940 0x7fa9680b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37350/0 (socket says 172.21.15.49:37350) 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 -- 172.21.15.49:0/2046679480 learned_addr learned my addr 172.21.15.49:0/2046679480 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96dced640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9680baef0 0x7fa9681dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96e4ee640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa96814e3b0 0x7fa9681e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 -- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa96814e3b0 msgr2=0x7fa9681e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa96814e3b0 0x7fa9681e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 -- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9680baef0 msgr2=0x7fa9681dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9680baef0 0x7fa9681dfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 -- 172.21.15.49:0/2046679480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa964009400 con 0x7fa9680bc940 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa96d4ec640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9680bc940 0x7fa9680b25d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa95800c490 tx=0x7fa95800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.137+0000 7fa95e7fc640 1 -- 172.21.15.49:0/2046679480 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9580052d0 con 0x7fa9680bc940 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.138+0000 7fa95e7fc640 1 -- 172.21.15.49:0/2046679480 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa958005430 con 0x7fa9680bc940 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.138+0000 7fa95e7fc640 1 -- 172.21.15.49:0/2046679480 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958012870 con 0x7fa9680bc940 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.138+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9681e0580 con 0x7fa9680bc940 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.138+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9681e0b40 con 0x7fa9680bc940 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.139+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa968066bb0 con 0x7fa9680bc940 2024-03-14T21:14:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.139+0000 7fa95e7fc640 1 -- 172.21.15.49:0/2046679480 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa9580129d0 con 0x7fa9680bc940 2024-03-14T21:14:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.139+0000 7fa95e7fc640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa94c06a7d0 0x7fa94c06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.139+0000 7fa95e7fc640 1 -- 172.21.15.49:0/2046679480 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa958088840 con 0x7fa9680bc940 2024-03-14T21:14:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.140+0000 7fa96dced640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa94c06a7d0 0x7fa94c06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.140+0000 7fa96dced640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa94c06a7d0 0x7fa94c06cc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa964036000 tx=0x7fa96403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.144+0000 7fa95e7fc640 1 -- 172.21.15.49:0/2046679480 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa958053d10 con 0x7fa9680bc940 2024-03-14T21:14:06.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.322+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9680bb9c0 con 0x7fa9680bc940 2024-03-14T21:14:06.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.324+0000 7fa95e7fc640 1 -- 172.21.15.49:0/2046679480 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fa958057330 con 0x7fa9680bc940 2024-03-14T21:14:06.307 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:06.307 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:04.014218+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:06.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa94c06a7d0 msgr2=0x7fa94c06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa94c06a7d0 0x7fa94c06cc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa964036000 tx=0x7fa96403c040 comp rx=0 tx=0).stop 2024-03-14T21:14:06.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9680bc940 msgr2=0x7fa9680b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9680bc940 0x7fa9680b25d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa95800c490 tx=0x7fa95800c950 comp rx=0 tx=0).stop 2024-03-14T21:14:06.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 shutdown_connections 2024-03-14T21:14:06.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa94c06a7d0 0x7fa94c06cc90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa96814e3b0 0x7fa9681e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9680bc940 0x7fa9680b25d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 --2- 172.21.15.49:0/2046679480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9680baef0 0x7fa9681dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 >> 172.21.15.49:0/2046679480 conn(0x7fa9680bcd80 msgr2=0x7fa9680b9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:06.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 shutdown_connections 2024-03-14T21:14:06.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.325+0000 7fa96ff51640 1 -- 172.21.15.49:0/2046679480 wait complete. 2024-03-14T21:14:06.319 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T21:14:06.319 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T21:14:06.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.442+0000 7faed6588640 1 Processor -- start 2024-03-14T21:14:06.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.442+0000 7faed6588640 1 -- start start 2024-03-14T21:14:06.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faed6588640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faed00bc940 0x7faed0156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faed6588640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed006a6e0 con 0x7faed00baef0 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faed6588640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00bc2b0 con 0x7faed006b620 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faed6588640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed006ad10 con 0x7faed00bc940 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faecffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7faed00baef0 0x7faed01480b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36818/0 (socket says 172.21.15.49:36818) 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faecffff640 1 -- 172.21.15.49:0/2142874157 learned_addr learned my addr 172.21.15.49:0/2142874157 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faed4b25640 1 --2- 172.21.15.49:0/2142874157 >> v2:172.21.15.97:3300/0 conn(0x7faed00bc940 0x7faed0156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758363058 0 0) 0x7faed006a6e0 con 0x7faed00baef0 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faebc003520 con 0x7faed00baef0 2024-03-14T21:14:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909155694 0 0) 0x7faebc003520 con 0x7faed00baef0 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 >> v2:172.21.15.97:3300/0 conn(0x7faed00bc940 msgr2=0x7faed0156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 --2- 172.21.15.49:0/2142874157 >> v2:172.21.15.97:3300/0 conn(0x7faed00bc940 0x7faed0156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 >> v1:172.21.15.97:6789/0 conn(0x7faed006b620 legacy=0x7faed00bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed006bab0 con 0x7faed00baef0 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7faeb8001670 con 0x7faed00baef0 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 >> v1:172.21.15.49:6789/0 conn(0x7faed00baef0 legacy=0x7faed01480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faebc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faebc0047c0 0x7faebc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faebc005180 0x7faebc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed006ad10 con 0x7faebc003970 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc00c120 con 0x7faebc005180 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc00c2e0 con 0x7faebc0047c0 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faecffff640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faebc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faecf7fe640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faebc0047c0 0x7faebc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.443+0000 7faed4b25640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faebc005180 0x7faebc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faecffff640 1 -- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faebc0047c0 msgr2=0x7faebc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faecffff640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faebc0047c0 0x7faebc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faecffff640 1 -- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faebc005180 msgr2=0x7faebc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faecffff640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faebc005180 0x7faebc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faecffff640 1 -- 172.21.15.49:0/2142874157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc003520 con 0x7faebc003970 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faed4b25640 1 -- 172.21.15.49:0/2142874157 reap_dead start 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faecffff640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faebc003d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c240 tx=0x7faebc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6657d0b6053127d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.444+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8005ad0 con 0x7faebc003970 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8005c30 con 0x7faebc003970 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faeceffd640 1 -- 172.21.15.49:0/2142874157 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb802ca90 con 0x7faebc003970 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faed6588640 1 -- 172.21.15.49:0/2142874157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 msgr2=0x7faebc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faed6588640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faebc003d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c240 tx=0x7faebc0057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faed6588640 1 -- 172.21.15.49:0/2142874157 shutdown_connections 2024-03-14T21:14:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faed6588640 1 --2- 172.21.15.49:0/2142874157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faebc003d90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faed6588640 1 -- 172.21.15.49:0/2142874157 >> 172.21.15.49:0/2142874157 conn(0x7faed00bcd80 msgr2=0x7faed00bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faed6588640 1 -- 172.21.15.49:0/2142874157 shutdown_connections 2024-03-14T21:14:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.445+0000 7faed6588640 1 -- 172.21.15.49:0/2142874157 wait complete. 2024-03-14T21:14:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.446+0000 7faed6588640 1 Processor -- start 2024-03-14T21:14:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.446+0000 7faed6588640 1 -- start start 2024-03-14T21:14:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faed6588640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faed00a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faed6588640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faed01df970 0x7faed00b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faed6588640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faed00b2bf0 0x7faed00a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faed6588640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed01e07f0 con 0x7faebc003970 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faed6588640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b30a0 con 0x7faed00b2bf0 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faed6588640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00a2740 con 0x7faed01df970 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faed00a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faed00a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37372/0 (socket says 172.21.15.49:37372) 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 -- 172.21.15.49:0/1050796795 learned_addr learned my addr 172.21.15.49:0/1050796795 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faed4b25640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faed00b2bf0 0x7faed00a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecf7fe640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faed01df970 0x7faed00b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 -- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faed01df970 msgr2=0x7faed00b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faed01df970 0x7faed00b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 -- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faed00b2bf0 msgr2=0x7faed00a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faed00b2bf0 0x7faed00a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 -- 172.21.15.49:0/1050796795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00b4270 con 0x7faebc003970 2024-03-14T21:14:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.447+0000 7faecffff640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faed00a2200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faeb8001930 tx=0x7faeb802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.448+0000 7faeccff9640 1 -- 172.21.15.49:0/1050796795 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb803c040 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.448+0000 7faeccff9640 1 -- 172.21.15.49:0/1050796795 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb802cd20 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.448+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed01dfe50 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.448+0000 7faeccff9640 1 -- 172.21.15.49:0/1050796795 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80072b0 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.448+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed01e0410 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.449+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed00666a0 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.449+0000 7faeccff9640 1 -- 172.21.15.49:0/1050796795 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7faeb8032070 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.449+0000 7faeccff9640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7faeb006a7d0 0x7faeb006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.450+0000 7faeccff9640 1 -- 172.21.15.49:0/1050796795 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faeb80aa230 con 0x7faebc003970 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.450+0000 7faecf7fe640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7faeb006a7d0 0x7faeb006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.450+0000 7faecf7fe640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7faeb006a7d0 0x7faeb006cc90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faed01dd580 tx=0x7faec0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.454+0000 7faeccff9640 1 -- 172.21.15.49:0/1050796795 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faeb80756e0 con 0x7faebc003970 2024-03-14T21:14:06.476 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T21:14:06.493+0000 7f137ad1f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T21:14:06.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.631+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faed00bb2d0 con 0x7faeb006a7d0 2024-03-14T21:14:06.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.637+0000 7faeccff9640 1 -- 172.21.15.49:0/1050796795 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249795 (secure 0 0 0) 0x7faec0001410 con 0x7faeb006a7d0 2024-03-14T21:14:06.624 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:06.626 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7faeb006a7d0 msgr2=0x7faeb006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7faeb006a7d0 0x7faeb006cc90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faed01dd580 tx=0x7faec0009040 comp rx=0 tx=0).stop 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 msgr2=0x7faed00a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faed00a2200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faeb8001930 tx=0x7faeb802e070 comp rx=0 tx=0).stop 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 shutdown_connections 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7faeb006a7d0 0x7faeb006cc90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faed00b2bf0 0x7faed00a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faed01df970 0x7faed00b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 --2- 172.21.15.49:0/1050796795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faebc003970 0x7faed00a2200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.640+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 >> 172.21.15.49:0/1050796795 conn(0x7faed00bcd80 msgr2=0x7faed00b9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.641+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 shutdown_connections 2024-03-14T21:14:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.641+0000 7faed6588640 1 -- 172.21.15.49:0/1050796795 wait complete. 2024-03-14T21:14:06.634 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-14T21:14:04.909278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":235272,"kb_used_data":2468,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754739448,"statfs":{"total":773094113280,"available":772853194752,"internally_reserved":0,"allocated":2527232,"data_stored":1583333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000970"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-03-14T21:13:54.136190+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T21:13:54.136190+0000","last_peered":"2024-03-14T21:13:54.136190+0000","last_clean":"2024-03-14T21:13:54.136190+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T21:13:54.136190+0000","last_undegraded":"2024-03-14T21:13:54.136190+0000","last_fullsized":"2024-03-14T21:13:54.136190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:02.638852+0000","last_change":"2024-03-14T21:14:02.638852+0000","last_active":"2024-03-14T21:14:02.638852+0000","last_peered":"2024-03-14T21:14:02.638852+0000","last_clean":"2024-03-14T21:14:02.638852+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:02.638852+0000","last_undegraded":"2024-03-14T21:14:02.638852+0000","last_fullsized":"2024-03-14T21:14:02.638852+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T21:14:01.634028+0000","last_change":"2024-03-14T21:14:01.634028+0000","last_active":"2024-03-14T21:14:01.634028+0000","last_peered":"2024-03-14T21:14:01.634028+0000","last_clean":"2024-03-14T21:14:01.634028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T21:14:01.634028+0000","last_undegraded":"2024-03-14T21:14:01.634028+0000","last_fullsized":"2024-03-14T21:14:01.634028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":757760,"data_stored":640151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T21:14:06.639 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T21:14:06.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.798+0000 7f8f24f2e640 1 Processor -- start 2024-03-14T21:14:06.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.798+0000 7f8f24f2e640 1 -- start start 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f24f2e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f24f2e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f2006b620 0x7f8f200bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f24f2e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f2006a6e0 con 0x7f8f200bc940 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f24f2e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2006ad10 con 0x7f8f2006b620 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f24f2e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200bc2b0 con 0x7f8f200baef0 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1dd9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f2006b620 0x7f8f200bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1ed9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8f200bc940 0x7f8f2014e3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36826/0 (socket says 172.21.15.49:36826) 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1dd9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f2006b620 0x7f8f200bf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37380/0 (socket says 172.21.15.49:37380) 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1ed9d640 1 -- 172.21.15.49:0/3070581148 learned_addr learned my addr 172.21.15.49:0/3070581148 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:06.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1e59c640 1 --2- 172.21.15.49:0/3070581148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1dd9b640 1 -- 172.21.15.49:0/3070581148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 msgr2=0x7f8f201480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1dd9b640 1 --2- 172.21.15.49:0/3070581148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1dd9b640 1 -- 172.21.15.49:0/3070581148 >> v1:172.21.15.49:6789/0 conn(0x7f8f200bc940 legacy=0x7f8f2014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1d59a640 1 -- 172.21.15.49:0/3070581148 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011780943 0 0) 0x7f8f2006a6e0 con 0x7f8f200bc940 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.799+0000 7f8f1dd9b640 1 -- 172.21.15.49:0/3070581148 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2006bab0 con 0x7f8f2006b620 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.800+0000 7f8f1dd9b640 1 --2- 172.21.15.49:0/3070581148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2006b620 0x7f8f200bf7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f0802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbec1f59a64ff290 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.800+0000 7f8f1d59a640 1 -- 172.21.15.49:0/3070581148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0802e070 con 0x7f8f2006b620 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.800+0000 7f8f1d59a640 1 -- 172.21.15.49:0/3070581148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f080315a0 con 0x7f8f2006b620 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.800+0000 7f8f1d59a640 1 -- 172.21.15.49:0/3070581148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08031930 con 0x7f8f2006b620 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 -- 172.21.15.49:0/3070581148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2006b620 msgr2=0x7f8f200bf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/3070581148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2006b620 0x7f8f200bf7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f0802b840 comp rx=0 tx=0).stop 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 -- 172.21.15.49:0/3070581148 shutdown_connections 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/3070581148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2006b620 0x7f8f200bf7f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/3070581148 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 -- 172.21.15.49:0/3070581148 >> 172.21.15.49:0/3070581148 conn(0x7f8f200bcd80 msgr2=0x7f8f200bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:06.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 -- 172.21.15.49:0/3070581148 shutdown_connections 2024-03-14T21:14:06.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.801+0000 7f8f24f2e640 1 -- 172.21.15.49:0/3070581148 wait complete. 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.802+0000 7f8f24f2e640 1 Processor -- start 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.802+0000 7f8f24f2e640 1 -- start start 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201dfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f200bc940 0x7f8f2014ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2014ef70 0x7f8f2009e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200b5a30 con 0x7f8f2014ef70 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200b48a0 con 0x7f8f200bc940 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200b4b40 con 0x7f8f200baef0 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2014ef70 0x7f8f2009e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2014ef70 0x7f8f2009e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37382/0 (socket says 172.21.15.49:37382) 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 -- 172.21.15.49:0/272522288 learned_addr learned my addr 172.21.15.49:0/272522288 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1e59c640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201dfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1dd9b640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f200bc940 0x7f8f2014ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 -- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 msgr2=0x7f8f201dfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201dfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 -- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f200bc940 msgr2=0x7f8f2014ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f200bc940 0x7f8f2014ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 -- 172.21.15.49:0/272522288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f08009400 con 0x7f8f2014ef70 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1dd9b640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f200bc940 0x7f8f2014ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f1ed9d640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2014ef70 0x7f8f2009e8b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000c730 tx=0x7f8f1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f16ffd640 1 -- 172.21.15.49:0/272522288 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000ce40 con 0x7f8f2014ef70 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f16ffd640 1 -- 172.21.15.49:0/272522288 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1000a640 con 0x7f8f2014ef70 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f200b4e20 con 0x7f8f2014ef70 2024-03-14T21:14:06.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f16ffd640 1 -- 172.21.15.49:0/272522288 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000aa30 con 0x7f8f2014ef70 2024-03-14T21:14:06.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.803+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2009f760 con 0x7f8f2014ef70 2024-03-14T21:14:06.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.805+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f20066bb0 con 0x7f8f2014ef70 2024-03-14T21:14:06.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.805+0000 7f8f16ffd640 1 -- 172.21.15.49:0/272522288 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8f1000ab90 con 0x7f8f2014ef70 2024-03-14T21:14:06.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.805+0000 7f8f16ffd640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f0006a7a0 0x7f8f0006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:06.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.805+0000 7f8f16ffd640 1 -- 172.21.15.49:0/272522288 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f100885a0 con 0x7f8f2014ef70 2024-03-14T21:14:06.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.806+0000 7f8f1e59c640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f0006a7a0 0x7f8f0006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:06.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.806+0000 7f8f1e59c640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f0006a7a0 0x7f8f0006cc60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009ad0 tx=0x7f8f18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:06.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.810+0000 7f8f16ffd640 1 -- 172.21.15.49:0/272522288 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8f10053930 con 0x7f8f2014ef70 2024-03-14T21:14:06.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.986+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f200bb2d0 con 0x7f8f0006a7a0 2024-03-14T21:14:06.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.993+0000 7f8f16ffd640 1 -- 172.21.15.49:0/272522288 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264655 (secure 0 0 0) 0x7f8f180119a0 con 0x7f8f0006a7a0 2024-03-14T21:14:06.981 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:06.983 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f0006a7a0 msgr2=0x7f8f0006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f0006a7a0 0x7f8f0006cc60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009ad0 tx=0x7f8f18008040 comp rx=0 tx=0).stop 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2014ef70 msgr2=0x7f8f2009e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2014ef70 0x7f8f2009e8b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000c730 tx=0x7f8f1000cbf0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 shutdown_connections 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f0006a7a0 0x7f8f0006cc60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f2014ef70 0x7f8f2009e8b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f200bc940 0x7f8f2014ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 --2- 172.21.15.49:0/272522288 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f200baef0 0x7f8f201dfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:06.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 >> 172.21.15.49:0/272522288 conn(0x7f8f200bcd80 msgr2=0x7f8f20072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:06.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 shutdown_connections 2024-03-14T21:14:06.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:06.996+0000 7f8f24f2e640 1 -- 172.21.15.49:0/272522288 wait complete. 2024-03-14T21:14:06.990 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-14T21:14:06.909961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235444,"kb_used_data":2868,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754739276,"statfs":{"total":773094113280,"available":772853018624,"internally_reserved":0,"allocated":2936832,"data_stored":1764193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.265789"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.032850+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T21:14:03.032850+0000","last_peered":"2024-03-14T21:14:03.032850+0000","last_clean":"2024-03-14T21:14:03.032850+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T21:14:03.032850+0000","last_undegraded":"2024-03-14T21:14:03.032850+0000","last_fullsized":"2024-03-14T21:14:03.032850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:57:09.325211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017662+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T21:14:04.017662+0000","last_peered":"2024-03-14T21:14:04.017662+0000","last_clean":"2024-03-14T21:14:04.017662+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T21:14:04.017662+0000","last_undegraded":"2024-03-14T21:14:04.017662+0000","last_fullsized":"2024-03-14T21:14:04.017662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.036452+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T21:14:03.036452+0000","last_peered":"2024-03-14T21:14:03.036452+0000","last_clean":"2024-03-14T21:14:03.036452+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T21:14:03.036452+0000","last_undegraded":"2024-03-14T21:14:03.036452+0000","last_fullsized":"2024-03-14T21:14:03.036452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:08:55.492956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019894+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T21:14:04.019894+0000","last_peered":"2024-03-14T21:14:04.019894+0000","last_clean":"2024-03-14T21:14:04.019894+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T21:14:04.019894+0000","last_undegraded":"2024-03-14T21:14:04.019894+0000","last_fullsized":"2024-03-14T21:14:04.019894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.037197+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T21:14:03.037197+0000","last_peered":"2024-03-14T21:14:03.037197+0000","last_clean":"2024-03-14T21:14:03.037197+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T21:14:03.037197+0000","last_undegraded":"2024-03-14T21:14:03.037197+0000","last_fullsized":"2024-03-14T21:14:03.037197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019354+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T21:14:04.019354+0000","last_peered":"2024-03-14T21:14:04.019354+0000","last_clean":"2024-03-14T21:14:04.019354+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T21:14:04.019354+0000","last_undegraded":"2024-03-14T21:14:04.019354+0000","last_fullsized":"2024-03-14T21:14:04.019354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.032802+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T21:14:03.032802+0000","last_peered":"2024-03-14T21:14:03.032802+0000","last_clean":"2024-03-14T21:14:03.032802+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T21:14:03.032802+0000","last_undegraded":"2024-03-14T21:14:03.032802+0000","last_fullsized":"2024-03-14T21:14:03.032802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243611+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T21:14:04.243611+0000","last_peered":"2024-03-14T21:14:04.243611+0000","last_clean":"2024-03-14T21:14:04.243611+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T21:14:04.243611+0000","last_undegraded":"2024-03-14T21:14:04.243611+0000","last_fullsized":"2024-03-14T21:14:04.243611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.032865+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T21:14:03.032865+0000","last_peered":"2024-03-14T21:14:03.032865+0000","last_clean":"2024-03-14T21:14:03.032865+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T21:14:03.032865+0000","last_undegraded":"2024-03-14T21:14:03.032865+0000","last_fullsized":"2024-03-14T21:14:03.032865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.040659+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T21:14:03.040659+0000","last_peered":"2024-03-14T21:14:03.040659+0000","last_clean":"2024-03-14T21:14:03.040659+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T21:14:03.040659+0000","last_undegraded":"2024-03-14T21:14:03.040659+0000","last_fullsized":"2024-03-14T21:14:03.040659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.040466+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T21:14:03.040466+0000","last_peered":"2024-03-14T21:14:03.040466+0000","last_clean":"2024-03-14T21:14:03.040466+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T21:14:03.040466+0000","last_undegraded":"2024-03-14T21:14:03.040466+0000","last_fullsized":"2024-03-14T21:14:03.040466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019337+0000","last_change":"2024-03-14T21:14:03.034043+0000","last_active":"2024-03-14T21:14:04.019337+0000","last_peered":"2024-03-14T21:14:04.019337+0000","last_clean":"2024-03-14T21:14:04.019337+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T21:14:04.019337+0000","last_undegraded":"2024-03-14T21:14:04.019337+0000","last_fullsized":"2024-03-14T21:14:04.019337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:33.787920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243838+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T21:14:04.243838+0000","last_peered":"2024-03-14T21:14:04.243838+0000","last_clean":"2024-03-14T21:14:04.243838+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T21:14:04.243838+0000","last_undegraded":"2024-03-14T21:14:04.243838+0000","last_fullsized":"2024-03-14T21:14:04.243838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.030928+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T21:14:03.030928+0000","last_peered":"2024-03-14T21:14:03.030928+0000","last_clean":"2024-03-14T21:14:03.030928+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T21:14:03.030928+0000","last_undegraded":"2024-03-14T21:14:03.030928+0000","last_fullsized":"2024-03-14T21:14:03.030928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017669+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T21:14:04.017669+0000","last_peered":"2024-03-14T21:14:04.017669+0000","last_clean":"2024-03-14T21:14:04.017669+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T21:14:04.017669+0000","last_undegraded":"2024-03-14T21:14:04.017669+0000","last_fullsized":"2024-03-14T21:14:04.017669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018774+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T21:14:04.018774+0000","last_peered":"2024-03-14T21:14:04.018774+0000","last_clean":"2024-03-14T21:14:04.018774+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T21:14:04.018774+0000","last_undegraded":"2024-03-14T21:14:04.018774+0000","last_fullsized":"2024-03-14T21:14:04.018774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019994+0000","last_change":"2024-03-14T21:14:03.035328+0000","last_active":"2024-03-14T21:14:04.019994+0000","last_peered":"2024-03-14T21:14:04.019994+0000","last_clean":"2024-03-14T21:14:04.019994+0000","last_became_active":"2024-03-14T21:14:03.034934+0000","last_became_peered":"2024-03-14T21:14:03.034934+0000","last_unstale":"2024-03-14T21:14:04.019994+0000","last_undegraded":"2024-03-14T21:14:04.019994+0000","last_fullsized":"2024-03-14T21:14:04.019994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.037535+0000","last_change":"2024-03-14T21:14:03.029823+0000","last_active":"2024-03-14T21:14:03.037535+0000","last_peered":"2024-03-14T21:14:03.037535+0000","last_clean":"2024-03-14T21:14:03.037535+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T21:14:03.037535+0000","last_undegraded":"2024-03-14T21:14:03.037535+0000","last_fullsized":"2024-03-14T21:14:03.037535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:47.131679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.037227+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T21:14:03.037227+0000","last_peered":"2024-03-14T21:14:03.037227+0000","last_clean":"2024-03-14T21:14:03.037227+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T21:14:03.037227+0000","last_undegraded":"2024-03-14T21:14:03.037227+0000","last_fullsized":"2024-03-14T21:14:03.037227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.040674+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T21:14:03.040674+0000","last_peered":"2024-03-14T21:14:03.040674+0000","last_clean":"2024-03-14T21:14:03.040674+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T21:14:03.040674+0000","last_undegraded":"2024-03-14T21:14:03.040674+0000","last_fullsized":"2024-03-14T21:14:03.040674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.036883+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T21:14:03.036883+0000","last_peered":"2024-03-14T21:14:03.036883+0000","last_clean":"2024-03-14T21:14:03.036883+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T21:14:03.036883+0000","last_undegraded":"2024-03-14T21:14:03.036883+0000","last_fullsized":"2024-03-14T21:14:03.036883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019386+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T21:14:04.019386+0000","last_peered":"2024-03-14T21:14:04.019386+0000","last_clean":"2024-03-14T21:14:04.019386+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T21:14:04.019386+0000","last_undegraded":"2024-03-14T21:14:04.019386+0000","last_fullsized":"2024-03-14T21:14:04.019386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243662+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T21:14:04.243662+0000","last_peered":"2024-03-14T21:14:04.243662+0000","last_clean":"2024-03-14T21:14:04.243662+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T21:14:04.243662+0000","last_undegraded":"2024-03-14T21:14:04.243662+0000","last_fullsized":"2024-03-14T21:14:04.243662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018867+0000","last_change":"2024-03-14T21:14:03.038694+0000","last_active":"2024-03-14T21:14:04.018867+0000","last_peered":"2024-03-14T21:14:04.018867+0000","last_clean":"2024-03-14T21:14:04.018867+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T21:14:04.018867+0000","last_undegraded":"2024-03-14T21:14:04.018867+0000","last_fullsized":"2024-03-14T21:14:04.018867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243990+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T21:14:04.243990+0000","last_peered":"2024-03-14T21:14:04.243990+0000","last_clean":"2024-03-14T21:14:04.243990+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T21:14:04.243990+0000","last_undegraded":"2024-03-14T21:14:04.243990+0000","last_fullsized":"2024-03-14T21:14:04.243990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.040725+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T21:14:03.040725+0000","last_peered":"2024-03-14T21:14:03.040725+0000","last_clean":"2024-03-14T21:14:03.040725+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T21:14:03.040725+0000","last_undegraded":"2024-03-14T21:14:03.040725+0000","last_fullsized":"2024-03-14T21:14:03.040725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017498+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T21:14:04.017498+0000","last_peered":"2024-03-14T21:14:04.017498+0000","last_clean":"2024-03-14T21:14:04.017498+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T21:14:04.017498+0000","last_undegraded":"2024-03-14T21:14:04.017498+0000","last_fullsized":"2024-03-14T21:14:04.017498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243915+0000","last_change":"2024-03-14T21:14:03.040690+0000","last_active":"2024-03-14T21:14:04.243915+0000","last_peered":"2024-03-14T21:14:04.243915+0000","last_clean":"2024-03-14T21:14:04.243915+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T21:14:04.243915+0000","last_undegraded":"2024-03-14T21:14:04.243915+0000","last_fullsized":"2024-03-14T21:14:04.243915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.039266+0000","last_change":"2024-03-14T21:14:03.038048+0000","last_active":"2024-03-14T21:14:03.039266+0000","last_peered":"2024-03-14T21:14:03.039266+0000","last_clean":"2024-03-14T21:14:03.039266+0000","last_became_active":"2024-03-14T21:14:03.037273+0000","last_became_peered":"2024-03-14T21:14:03.037273+0000","last_unstale":"2024-03-14T21:14:03.039266+0000","last_undegraded":"2024-03-14T21:14:03.039266+0000","last_fullsized":"2024-03-14T21:14:03.039266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019469+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T21:14:04.019469+0000","last_peered":"2024-03-14T21:14:04.019469+0000","last_clean":"2024-03-14T21:14:04.019469+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T21:14:04.019469+0000","last_undegraded":"2024-03-14T21:14:04.019469+0000","last_fullsized":"2024-03-14T21:14:04.019469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.037461+0000","last_change":"2024-03-14T21:14:03.029847+0000","last_active":"2024-03-14T21:14:03.037461+0000","last_peered":"2024-03-14T21:14:03.037461+0000","last_clean":"2024-03-14T21:14:03.037461+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T21:14:03.037461+0000","last_undegraded":"2024-03-14T21:14:03.037461+0000","last_fullsized":"2024-03-14T21:14:03.037461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.040719+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T21:14:03.040719+0000","last_peered":"2024-03-14T21:14:03.040719+0000","last_clean":"2024-03-14T21:14:03.040719+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T21:14:03.040719+0000","last_undegraded":"2024-03-14T21:14:03.040719+0000","last_fullsized":"2024-03-14T21:14:03.040719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018978+0000","last_change":"2024-03-14T21:14:03.037915+0000","last_active":"2024-03-14T21:14:04.018978+0000","last_peered":"2024-03-14T21:14:04.018978+0000","last_clean":"2024-03-14T21:14:04.018978+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T21:14:04.018978+0000","last_undegraded":"2024-03-14T21:14:04.018978+0000","last_fullsized":"2024-03-14T21:14:04.018978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243993+0000","last_change":"2024-03-14T21:14:02.655762+0000","last_active":"2024-03-14T21:14:04.243993+0000","last_peered":"2024-03-14T21:14:04.243993+0000","last_clean":"2024-03-14T21:14:04.243993+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T21:14:04.243993+0000","last_undegraded":"2024-03-14T21:14:04.243993+0000","last_fullsized":"2024-03-14T21:14:04.243993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.035849+0000","last_change":"2024-03-14T21:14:03.033091+0000","last_active":"2024-03-14T21:14:03.035849+0000","last_peered":"2024-03-14T21:14:03.035849+0000","last_clean":"2024-03-14T21:14:03.035849+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T21:14:03.035849+0000","last_undegraded":"2024-03-14T21:14:03.035849+0000","last_fullsized":"2024-03-14T21:14:03.035849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.026258+0000","last_change":"2024-03-14T21:14:02.662899+0000","last_active":"2024-03-14T21:14:03.026258+0000","last_peered":"2024-03-14T21:14:03.026258+0000","last_clean":"2024-03-14T21:14:03.026258+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T21:14:03.026258+0000","last_undegraded":"2024-03-14T21:14:03.026258+0000","last_fullsized":"2024-03-14T21:14:03.026258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243862+0000","last_change":"2024-03-14T21:14:02.660574+0000","last_active":"2024-03-14T21:14:04.243862+0000","last_peered":"2024-03-14T21:14:04.243862+0000","last_clean":"2024-03-14T21:14:04.243862+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T21:14:04.243862+0000","last_undegraded":"2024-03-14T21:14:04.243862+0000","last_fullsized":"2024-03-14T21:14:04.243862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243590+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T21:14:04.243590+0000","last_peered":"2024-03-14T21:14:04.243590+0000","last_clean":"2024-03-14T21:14:04.243590+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T21:14:04.243590+0000","last_undegraded":"2024-03-14T21:14:04.243590+0000","last_fullsized":"2024-03-14T21:14:04.243590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018961+0000","last_change":"2024-03-14T21:14:02.661303+0000","last_active":"2024-03-14T21:14:04.018961+0000","last_peered":"2024-03-14T21:14:04.018961+0000","last_clean":"2024-03-14T21:14:04.018961+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T21:14:04.018961+0000","last_undegraded":"2024-03-14T21:14:04.018961+0000","last_fullsized":"2024-03-14T21:14:04.018961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017394+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T21:14:04.017394+0000","last_peered":"2024-03-14T21:14:04.017394+0000","last_clean":"2024-03-14T21:14:04.017394+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T21:14:04.017394+0000","last_undegraded":"2024-03-14T21:14:04.017394+0000","last_fullsized":"2024-03-14T21:14:04.017394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.022224+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T21:14:03.022224+0000","last_peered":"2024-03-14T21:14:03.022224+0000","last_clean":"2024-03-14T21:14:03.022224+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T21:14:03.022224+0000","last_undegraded":"2024-03-14T21:14:03.022224+0000","last_fullsized":"2024-03-14T21:14:03.022224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019922+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T21:14:04.019922+0000","last_peered":"2024-03-14T21:14:04.019922+0000","last_clean":"2024-03-14T21:14:04.019922+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T21:14:04.019922+0000","last_undegraded":"2024-03-14T21:14:04.019922+0000","last_fullsized":"2024-03-14T21:14:04.019922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019816+0000","last_change":"2024-03-14T21:14:02.660384+0000","last_active":"2024-03-14T21:14:04.019816+0000","last_peered":"2024-03-14T21:14:04.019816+0000","last_clean":"2024-03-14T21:14:04.019816+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T21:14:04.019816+0000","last_undegraded":"2024-03-14T21:14:04.019816+0000","last_fullsized":"2024-03-14T21:14:04.019816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.035638+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T21:14:03.035638+0000","last_peered":"2024-03-14T21:14:03.035638+0000","last_clean":"2024-03-14T21:14:03.035638+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T21:14:03.035638+0000","last_undegraded":"2024-03-14T21:14:03.035638+0000","last_fullsized":"2024-03-14T21:14:03.035638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:57:33.155885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017568+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T21:14:04.017568+0000","last_peered":"2024-03-14T21:14:04.017568+0000","last_clean":"2024-03-14T21:14:04.017568+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T21:14:04.017568+0000","last_undegraded":"2024-03-14T21:14:04.017568+0000","last_fullsized":"2024-03-14T21:14:04.017568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019326+0000","last_change":"2024-03-14T21:14:03.032533+0000","last_active":"2024-03-14T21:14:04.019326+0000","last_peered":"2024-03-14T21:14:04.019326+0000","last_clean":"2024-03-14T21:14:04.019326+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T21:14:04.019326+0000","last_undegraded":"2024-03-14T21:14:04.019326+0000","last_fullsized":"2024-03-14T21:14:04.019326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:41.581386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.047709+0000","last_change":"2024-03-14T21:14:02.659028+0000","last_active":"2024-03-14T21:14:03.047709+0000","last_peered":"2024-03-14T21:14:03.047709+0000","last_clean":"2024-03-14T21:14:03.047709+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T21:14:03.047709+0000","last_undegraded":"2024-03-14T21:14:03.047709+0000","last_fullsized":"2024-03-14T21:14:03.047709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.035547+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T21:14:03.035547+0000","last_peered":"2024-03-14T21:14:03.035547+0000","last_clean":"2024-03-14T21:14:03.035547+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T21:14:03.035547+0000","last_undegraded":"2024-03-14T21:14:03.035547+0000","last_fullsized":"2024-03-14T21:14:03.035547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.037811+0000","last_change":"2024-03-14T21:14:02.663139+0000","last_active":"2024-03-14T21:14:03.037811+0000","last_peered":"2024-03-14T21:14:03.037811+0000","last_clean":"2024-03-14T21:14:03.037811+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T21:14:03.037811+0000","last_undegraded":"2024-03-14T21:14:03.037811+0000","last_fullsized":"2024-03-14T21:14:03.037811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:25:31.095452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.036816+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T21:14:03.036816+0000","last_peered":"2024-03-14T21:14:03.036816+0000","last_clean":"2024-03-14T21:14:03.036816+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T21:14:03.036816+0000","last_undegraded":"2024-03-14T21:14:03.036816+0000","last_fullsized":"2024-03-14T21:14:03.036816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019604+0000","last_change":"2024-03-14T21:14:02.659773+0000","last_active":"2024-03-14T21:14:04.019604+0000","last_peered":"2024-03-14T21:14:04.019604+0000","last_clean":"2024-03-14T21:14:04.019604+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T21:14:04.019604+0000","last_undegraded":"2024-03-14T21:14:04.019604+0000","last_fullsized":"2024-03-14T21:14:04.019604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.035568+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T21:14:03.035568+0000","last_peered":"2024-03-14T21:14:03.035568+0000","last_clean":"2024-03-14T21:14:03.035568+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T21:14:03.035568+0000","last_undegraded":"2024-03-14T21:14:03.035568+0000","last_fullsized":"2024-03-14T21:14:03.035568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.020757+0000","last_change":"2024-03-14T21:14:02.663585+0000","last_active":"2024-03-14T21:14:03.020757+0000","last_peered":"2024-03-14T21:14:03.020757+0000","last_clean":"2024-03-14T21:14:03.020757+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T21:14:03.020757+0000","last_undegraded":"2024-03-14T21:14:03.020757+0000","last_fullsized":"2024-03-14T21:14:03.020757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.040698+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T21:14:03.040698+0000","last_peered":"2024-03-14T21:14:03.040698+0000","last_clean":"2024-03-14T21:14:03.040698+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T21:14:03.040698+0000","last_undegraded":"2024-03-14T21:14:03.040698+0000","last_fullsized":"2024-03-14T21:14:03.040698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.033708+0000","last_change":"2024-03-14T21:14:02.654931+0000","last_active":"2024-03-14T21:14:03.033708+0000","last_peered":"2024-03-14T21:14:03.033708+0000","last_clean":"2024-03-14T21:14:03.033708+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T21:14:03.033708+0000","last_undegraded":"2024-03-14T21:14:03.033708+0000","last_fullsized":"2024-03-14T21:14:03.033708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.042685+0000","last_change":"2024-03-14T21:14:03.037873+0000","last_active":"2024-03-14T21:14:03.042685+0000","last_peered":"2024-03-14T21:14:03.042685+0000","last_clean":"2024-03-14T21:14:03.042685+0000","last_became_active":"2024-03-14T21:14:03.037608+0000","last_became_peered":"2024-03-14T21:14:03.037608+0000","last_unstale":"2024-03-14T21:14:03.042685+0000","last_undegraded":"2024-03-14T21:14:03.042685+0000","last_fullsized":"2024-03-14T21:14:03.042685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:46:32.269714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019235+0000","last_change":"2024-03-14T21:14:02.663747+0000","last_active":"2024-03-14T21:14:04.019235+0000","last_peered":"2024-03-14T21:14:04.019235+0000","last_clean":"2024-03-14T21:14:04.019235+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T21:14:04.019235+0000","last_undegraded":"2024-03-14T21:14:04.019235+0000","last_fullsized":"2024-03-14T21:14:04.019235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:59:11.091842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.032820+0000","last_change":"2024-03-14T21:14:03.030867+0000","last_active":"2024-03-14T21:14:03.032820+0000","last_peered":"2024-03-14T21:14:03.032820+0000","last_clean":"2024-03-14T21:14:03.032820+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T21:14:03.032820+0000","last_undegraded":"2024-03-14T21:14:03.032820+0000","last_fullsized":"2024-03-14T21:14:03.032820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.036755+0000","last_change":"2024-03-14T21:14:02.659095+0000","last_active":"2024-03-14T21:14:03.036755+0000","last_peered":"2024-03-14T21:14:03.036755+0000","last_clean":"2024-03-14T21:14:03.036755+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T21:14:03.036755+0000","last_undegraded":"2024-03-14T21:14:03.036755+0000","last_fullsized":"2024-03-14T21:14:03.036755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.042699+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T21:14:03.042699+0000","last_peered":"2024-03-14T21:14:03.042699+0000","last_clean":"2024-03-14T21:14:03.042699+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T21:14:03.042699+0000","last_undegraded":"2024-03-14T21:14:03.042699+0000","last_fullsized":"2024-03-14T21:14:03.042699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017595+0000","last_change":"2024-03-14T21:14:02.663441+0000","last_active":"2024-03-14T21:14:04.017595+0000","last_peered":"2024-03-14T21:14:04.017595+0000","last_clean":"2024-03-14T21:14:04.017595+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T21:14:04.017595+0000","last_undegraded":"2024-03-14T21:14:04.017595+0000","last_fullsized":"2024-03-14T21:14:04.017595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.020037+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T21:14:04.020037+0000","last_peered":"2024-03-14T21:14:04.020037+0000","last_clean":"2024-03-14T21:14:04.020037+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T21:14:04.020037+0000","last_undegraded":"2024-03-14T21:14:04.020037+0000","last_fullsized":"2024-03-14T21:14:04.020037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.020418+0000","last_change":"2024-03-14T21:14:02.653919+0000","last_active":"2024-03-14T21:14:03.020418+0000","last_peered":"2024-03-14T21:14:03.020418+0000","last_clean":"2024-03-14T21:14:03.020418+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T21:14:03.020418+0000","last_undegraded":"2024-03-14T21:14:03.020418+0000","last_fullsized":"2024-03-14T21:14:03.020418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.032832+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T21:14:03.032832+0000","last_peered":"2024-03-14T21:14:03.032832+0000","last_clean":"2024-03-14T21:14:03.032832+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T21:14:03.032832+0000","last_undegraded":"2024-03-14T21:14:03.032832+0000","last_fullsized":"2024-03-14T21:14:03.032832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-14T21:14:02.652802+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T21:14:02.652802+0000","last_peered":"2024-03-14T21:14:02.652802+0000","last_clean":"2024-03-14T21:14:02.652802+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T21:14:02.652802+0000","last_undegraded":"2024-03-14T21:14:02.652802+0000","last_fullsized":"2024-03-14T21:14:02.652802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019557+0000","last_change":"2024-03-14T21:14:02.652881+0000","last_active":"2024-03-14T21:14:04.019557+0000","last_peered":"2024-03-14T21:14:04.019557+0000","last_clean":"2024-03-14T21:14:04.019557+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T21:14:04.019557+0000","last_undegraded":"2024-03-14T21:14:04.019557+0000","last_fullsized":"2024-03-14T21:14:04.019557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243730+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T21:14:04.243730+0000","last_peered":"2024-03-14T21:14:04.243730+0000","last_clean":"2024-03-14T21:14:04.243730+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T21:14:04.243730+0000","last_undegraded":"2024-03-14T21:14:04.243730+0000","last_fullsized":"2024-03-14T21:14:04.243730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.020345+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T21:14:03.020345+0000","last_peered":"2024-03-14T21:14:03.020345+0000","last_clean":"2024-03-14T21:14:03.020345+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T21:14:03.020345+0000","last_undegraded":"2024-03-14T21:14:03.020345+0000","last_fullsized":"2024-03-14T21:14:03.020345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:46.126855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.035807+0000","last_change":"2024-03-14T21:14:03.033195+0000","last_active":"2024-03-14T21:14:03.035807+0000","last_peered":"2024-03-14T21:14:03.035807+0000","last_clean":"2024-03-14T21:14:03.035807+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T21:14:03.035807+0000","last_undegraded":"2024-03-14T21:14:03.035807+0000","last_fullsized":"2024-03-14T21:14:03.035807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.020246+0000","last_change":"2024-03-14T21:14:02.658709+0000","last_active":"2024-03-14T21:14:03.020246+0000","last_peered":"2024-03-14T21:14:03.020246+0000","last_clean":"2024-03-14T21:14:03.020246+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T21:14:03.020246+0000","last_undegraded":"2024-03-14T21:14:03.020246+0000","last_fullsized":"2024-03-14T21:14:03.020246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017391+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T21:14:04.017391+0000","last_peered":"2024-03-14T21:14:04.017391+0000","last_clean":"2024-03-14T21:14:04.017391+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T21:14:04.017391+0000","last_undegraded":"2024-03-14T21:14:04.017391+0000","last_fullsized":"2024-03-14T21:14:04.017391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019554+0000","last_change":"2024-03-14T21:14:02.657401+0000","last_active":"2024-03-14T21:14:04.019554+0000","last_peered":"2024-03-14T21:14:04.019554+0000","last_clean":"2024-03-14T21:14:04.019554+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T21:14:04.019554+0000","last_undegraded":"2024-03-14T21:14:04.019554+0000","last_fullsized":"2024-03-14T21:14:04.019554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019342+0000","last_change":"2024-03-14T21:14:03.036152+0000","last_active":"2024-03-14T21:14:04.019342+0000","last_peered":"2024-03-14T21:14:04.019342+0000","last_clean":"2024-03-14T21:14:04.019342+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T21:14:04.019342+0000","last_undegraded":"2024-03-14T21:14:04.019342+0000","last_fullsized":"2024-03-14T21:14:04.019342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:14.794102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017565+0000","last_change":"2024-03-14T21:14:02.653065+0000","last_active":"2024-03-14T21:14:04.017565+0000","last_peered":"2024-03-14T21:14:04.017565+0000","last_clean":"2024-03-14T21:14:04.017565+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T21:14:04.017565+0000","last_undegraded":"2024-03-14T21:14:04.017565+0000","last_fullsized":"2024-03-14T21:14:04.017565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.020019+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T21:14:04.020019+0000","last_peered":"2024-03-14T21:14:04.020019+0000","last_clean":"2024-03-14T21:14:04.020019+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T21:14:04.020019+0000","last_undegraded":"2024-03-14T21:14:04.020019+0000","last_fullsized":"2024-03-14T21:14:04.020019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.022956+0000","last_change":"2024-03-14T21:14:02.659612+0000","last_active":"2024-03-14T21:14:03.022956+0000","last_peered":"2024-03-14T21:14:03.022956+0000","last_clean":"2024-03-14T21:14:03.022956+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T21:14:03.022956+0000","last_undegraded":"2024-03-14T21:14:03.022956+0000","last_fullsized":"2024-03-14T21:14:03.022956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017470+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T21:14:04.017470+0000","last_peered":"2024-03-14T21:14:04.017470+0000","last_clean":"2024-03-14T21:14:04.017470+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T21:14:04.017470+0000","last_undegraded":"2024-03-14T21:14:04.017470+0000","last_fullsized":"2024-03-14T21:14:04.017470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017666+0000","last_change":"2024-03-14T21:14:02.662122+0000","last_active":"2024-03-14T21:14:04.017666+0000","last_peered":"2024-03-14T21:14:04.017666+0000","last_clean":"2024-03-14T21:14:04.017666+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T21:14:04.017666+0000","last_undegraded":"2024-03-14T21:14:04.017666+0000","last_fullsized":"2024-03-14T21:14:04.017666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.244002+0000","last_change":"2024-03-14T21:14:03.037921+0000","last_active":"2024-03-14T21:14:04.244002+0000","last_peered":"2024-03-14T21:14:04.244002+0000","last_clean":"2024-03-14T21:14:04.244002+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T21:14:04.244002+0000","last_undegraded":"2024-03-14T21:14:04.244002+0000","last_fullsized":"2024-03-14T21:14:04.244002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.038187+0000","last_change":"2024-03-14T21:14:02.665745+0000","last_active":"2024-03-14T21:14:03.038187+0000","last_peered":"2024-03-14T21:14:03.038187+0000","last_clean":"2024-03-14T21:14:03.038187+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T21:14:03.038187+0000","last_undegraded":"2024-03-14T21:14:03.038187+0000","last_fullsized":"2024-03-14T21:14:03.038187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.032438+0000","last_change":"2024-03-14T21:14:03.029269+0000","last_active":"2024-03-14T21:14:03.032438+0000","last_peered":"2024-03-14T21:14:03.032438+0000","last_clean":"2024-03-14T21:14:03.032438+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T21:14:03.032438+0000","last_undegraded":"2024-03-14T21:14:03.032438+0000","last_fullsized":"2024-03-14T21:14:03.032438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.020247+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T21:14:03.020247+0000","last_peered":"2024-03-14T21:14:03.020247+0000","last_clean":"2024-03-14T21:14:03.020247+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T21:14:03.020247+0000","last_undegraded":"2024-03-14T21:14:03.020247+0000","last_fullsized":"2024-03-14T21:14:03.020247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019008+0000","last_change":"2024-03-14T21:14:03.039549+0000","last_active":"2024-03-14T21:14:04.019008+0000","last_peered":"2024-03-14T21:14:04.019008+0000","last_clean":"2024-03-14T21:14:04.019008+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T21:14:04.019008+0000","last_undegraded":"2024-03-14T21:14:04.019008+0000","last_fullsized":"2024-03-14T21:14:04.019008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:23.510516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017601+0000","last_change":"2024-03-14T21:14:02.660292+0000","last_active":"2024-03-14T21:14:04.017601+0000","last_peered":"2024-03-14T21:14:04.017601+0000","last_clean":"2024-03-14T21:14:04.017601+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T21:14:04.017601+0000","last_undegraded":"2024-03-14T21:14:04.017601+0000","last_fullsized":"2024-03-14T21:14:04.017601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243693+0000","last_change":"2024-03-14T21:14:03.040817+0000","last_active":"2024-03-14T21:14:04.243693+0000","last_peered":"2024-03-14T21:14:04.243693+0000","last_clean":"2024-03-14T21:14:04.243693+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T21:14:04.243693+0000","last_undegraded":"2024-03-14T21:14:04.243693+0000","last_fullsized":"2024-03-14T21:14:04.243693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017553+0000","last_change":"2024-03-14T21:14:02.661073+0000","last_active":"2024-03-14T21:14:04.017553+0000","last_peered":"2024-03-14T21:14:04.017553+0000","last_clean":"2024-03-14T21:14:04.017553+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T21:14:04.017553+0000","last_undegraded":"2024-03-14T21:14:04.017553+0000","last_fullsized":"2024-03-14T21:14:04.017553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243996+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T21:14:04.243996+0000","last_peered":"2024-03-14T21:14:04.243996+0000","last_clean":"2024-03-14T21:14:04.243996+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T21:14:04.243996+0000","last_undegraded":"2024-03-14T21:14:04.243996+0000","last_fullsized":"2024-03-14T21:14:04.243996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.033420+0000","last_change":"2024-03-14T21:14:02.663131+0000","last_active":"2024-03-14T21:14:03.033420+0000","last_peered":"2024-03-14T21:14:03.033420+0000","last_clean":"2024-03-14T21:14:03.033420+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T21:14:03.033420+0000","last_undegraded":"2024-03-14T21:14:03.033420+0000","last_fullsized":"2024-03-14T21:14:03.033420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.042846+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T21:14:03.042846+0000","last_peered":"2024-03-14T21:14:03.042846+0000","last_clean":"2024-03-14T21:14:03.042846+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T21:14:03.042846+0000","last_undegraded":"2024-03-14T21:14:03.042846+0000","last_fullsized":"2024-03-14T21:14:03.042846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019966+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T21:14:04.019966+0000","last_peered":"2024-03-14T21:14:04.019966+0000","last_clean":"2024-03-14T21:14:04.019966+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T21:14:04.019966+0000","last_undegraded":"2024-03-14T21:14:04.019966+0000","last_fullsized":"2024-03-14T21:14:04.019966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.031402+0000","last_change":"2024-03-14T21:14:02.660393+0000","last_active":"2024-03-14T21:14:03.031402+0000","last_peered":"2024-03-14T21:14:03.031402+0000","last_clean":"2024-03-14T21:14:03.031402+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T21:14:03.031402+0000","last_undegraded":"2024-03-14T21:14:03.031402+0000","last_fullsized":"2024-03-14T21:14:03.031402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019982+0000","last_change":"2024-03-14T21:14:03.035149+0000","last_active":"2024-03-14T21:14:04.019982+0000","last_peered":"2024-03-14T21:14:04.019982+0000","last_clean":"2024-03-14T21:14:04.019982+0000","last_became_active":"2024-03-14T21:14:03.034649+0000","last_became_peered":"2024-03-14T21:14:03.034649+0000","last_unstale":"2024-03-14T21:14:04.019982+0000","last_undegraded":"2024-03-14T21:14:04.019982+0000","last_fullsized":"2024-03-14T21:14:04.019982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.039252+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T21:14:03.039252+0000","last_peered":"2024-03-14T21:14:03.039252+0000","last_clean":"2024-03-14T21:14:03.039252+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T21:14:03.039252+0000","last_undegraded":"2024-03-14T21:14:03.039252+0000","last_fullsized":"2024-03-14T21:14:03.039252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:48.203059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.038972+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T21:14:03.038972+0000","last_peered":"2024-03-14T21:14:03.038972+0000","last_clean":"2024-03-14T21:14:03.038972+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T21:14:03.038972+0000","last_undegraded":"2024-03-14T21:14:03.038972+0000","last_fullsized":"2024-03-14T21:14:03.038972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243988+0000","last_change":"2024-03-14T21:14:02.654963+0000","last_active":"2024-03-14T21:14:04.243988+0000","last_peered":"2024-03-14T21:14:04.243988+0000","last_clean":"2024-03-14T21:14:04.243988+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T21:14:04.243988+0000","last_undegraded":"2024-03-14T21:14:04.243988+0000","last_fullsized":"2024-03-14T21:14:04.243988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243950+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T21:14:04.243950+0000","last_peered":"2024-03-14T21:14:04.243950+0000","last_clean":"2024-03-14T21:14:04.243950+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T21:14:04.243950+0000","last_undegraded":"2024-03-14T21:14:04.243950+0000","last_fullsized":"2024-03-14T21:14:04.243950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.035556+0000","last_change":"2024-03-14T21:14:02.656125+0000","last_active":"2024-03-14T21:14:03.035556+0000","last_peered":"2024-03-14T21:14:03.035556+0000","last_clean":"2024-03-14T21:14:03.035556+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T21:14:03.035556+0000","last_undegraded":"2024-03-14T21:14:03.035556+0000","last_fullsized":"2024-03-14T21:14:03.035556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":204800,"data_stored":69269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":806912,"data_stored":659656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T21:14:06.994 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T21:14:06.995 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-14T21:14:06.995 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-14T21:14:06.995 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-14T21:14:06.995 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-14T21:14:07.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.153+0000 7f2568cc4640 1 Processor -- start 2024-03-14T21:14:07.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.153+0000 7f2568cc4640 1 -- start start 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2568cc4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25640bc940 0x7f2564156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2568cc4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256406a6e0 con 0x7f25640baef0 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2568cc4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25640bc2b0 con 0x7f256406b620 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2568cc4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256406ad10 con 0x7f25640bc940 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2562d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25640bc940 0x7f2564156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f256259c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f25640baef0 0x7f25641480b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36834/0 (socket says 172.21.15.49:36834) 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2562d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25640bc940 0x7f2564156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37398/0 (socket says 172.21.15.49:37398) 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f256259c640 1 -- 172.21.15.49:0/1018236673 learned_addr learned my addr 172.21.15.49:0/1018236673 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2562d9d640 1 -- 172.21.15.49:0/1018236673 >> v1:172.21.15.97:6789/0 conn(0x7f256406b620 legacy=0x7f25640bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2562d9d640 1 -- 172.21.15.49:0/1018236673 >> v1:172.21.15.49:6789/0 conn(0x7f25640baef0 legacy=0x7f25641480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f2562d9d640 1 -- 172.21.15.49:0/1018236673 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256406bab0 con 0x7f25640bc940 2024-03-14T21:14:07.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.154+0000 7f256159a640 1 -- 172.21.15.49:0/1018236673 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321457625 0 0) 0x7f256406a6e0 con 0x7f25640baef0 2024-03-14T21:14:07.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.155+0000 7f2562d9d640 1 --2- 172.21.15.49:0/1018236673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25640bc940 0x7f2564156720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f255402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cdb840d0c1956b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.155+0000 7f256159a640 1 -- 172.21.15.49:0/1018236673 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255402cbd0 con 0x7f25640bc940 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.155+0000 7f256159a640 1 -- 172.21.15.49:0/1018236673 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255402cd30 con 0x7f25640bc940 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.155+0000 7f256159a640 1 -- 172.21.15.49:0/1018236673 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255402cea0 con 0x7f25640bc940 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.156+0000 7f2568cc4640 1 -- 172.21.15.49:0/1018236673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25640bc940 msgr2=0x7f2564156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.156+0000 7f2568cc4640 1 --2- 172.21.15.49:0/1018236673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25640bc940 0x7f2564156720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f255402be20 comp rx=0 tx=0).stop 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.156+0000 7f2568cc4640 1 -- 172.21.15.49:0/1018236673 shutdown_connections 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.156+0000 7f2568cc4640 1 --2- 172.21.15.49:0/1018236673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25640bc940 0x7f2564156720 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.156+0000 7f2568cc4640 1 -- 172.21.15.49:0/1018236673 >> 172.21.15.49:0/1018236673 conn(0x7f25640bcd80 msgr2=0x7f25640bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.156+0000 7f2568cc4640 1 -- 172.21.15.49:0/1018236673 shutdown_connections 2024-03-14T21:14:07.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.156+0000 7f2568cc4640 1 -- 172.21.15.49:0/1018236673 wait complete. 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.157+0000 7f2568cc4640 1 Processor -- start 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.157+0000 7f2568cc4640 1 -- start start 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25640baef0 0x7f256409a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25640bc940 0x7f25641d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f256409b0b0 0x7f25641dff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256409a620 con 0x7f256409b0b0 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641d1900 con 0x7f25640baef0 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256409f670 con 0x7f25640bc940 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f256409b0b0 0x7f25641dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f256409b0b0 0x7f25641dff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37400/0 (socket says 172.21.15.49:37400) 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 -- 172.21.15.49:0/2327284811 learned_addr learned my addr 172.21.15.49:0/2327284811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f256259c640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25640baef0 0x7f256409a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2561d9b640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25640bc940 0x7f25641d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 -- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25640bc940 msgr2=0x7f25641d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25640bc940 0x7f25641d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 -- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25640baef0 msgr2=0x7f256409a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25640baef0 0x7f256409a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 -- 172.21.15.49:0/2327284811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2554009a70 con 0x7f256409b0b0 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2562d9d640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f256409b0b0 0x7f25641dff70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2554009eb0 tx=0x7f2554004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2552ffd640 1 -- 172.21.15.49:0/2327284811 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255402cea0 con 0x7f256409b0b0 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2552ffd640 1 -- 172.21.15.49:0/2327284811 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255403f420 con 0x7f256409b0b0 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2564148940 con 0x7f256409b0b0 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2552ffd640 1 -- 172.21.15.49:0/2327284811 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255403f790 con 0x7f256409b0b0 2024-03-14T21:14:07.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.158+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25641e35f0 con 0x7f256409b0b0 2024-03-14T21:14:07.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.159+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25640733c0 con 0x7f256409b0b0 2024-03-14T21:14:07.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.160+0000 7f2552ffd640 1 -- 172.21.15.49:0/2327284811 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f255403f8f0 con 0x7f256409b0b0 2024-03-14T21:14:07.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.160+0000 7f2552ffd640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f254406a7a0 0x7f254406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.160+0000 7f2552ffd640 1 -- 172.21.15.49:0/2327284811 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25540ab950 con 0x7f256409b0b0 2024-03-14T21:14:07.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.160+0000 7f256259c640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f254406a7a0 0x7f254406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.161+0000 7f256259c640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f254406a7a0 0x7f254406cc60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f255c000f60 tx=0x7f255c007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.165+0000 7f2552ffd640 1 -- 172.21.15.49:0/2327284811 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2554076e00 con 0x7f256409b0b0 2024-03-14T21:14:07.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.394+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f25640666a0 con 0x7f256409b0b0 2024-03-14T21:14:07.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.397+0000 7f2552ffd640 1 -- 172.21.15.49:0/2327284811 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170389 (secure 0 0 0) 0x7f2554043090 con 0x7f256409b0b0 2024-03-14T21:14:07.383 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:07.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f254406a7a0 msgr2=0x7f254406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f254406a7a0 0x7f254406cc60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f255c000f60 tx=0x7f255c007b50 comp rx=0 tx=0).stop 2024-03-14T21:14:07.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f256409b0b0 msgr2=0x7f25641dff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f256409b0b0 0x7f25641dff70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2554009eb0 tx=0x7f2554004a40 comp rx=0 tx=0).stop 2024-03-14T21:14:07.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 shutdown_connections 2024-03-14T21:14:07.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f254406a7a0 0x7f254406cc60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f256409b0b0 0x7f25641dff70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f25640bc940 0x7f25641d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 --2- 172.21.15.49:0/2327284811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25640baef0 0x7f256409a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 >> 172.21.15.49:0/2327284811 conn(0x7f25640bcd80 msgr2=0x7f2564072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:07.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 shutdown_connections 2024-03-14T21:14:07.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.400+0000 7f2568cc4640 1 -- 172.21.15.49:0/2327284811 wait complete. 2024-03-14T21:14:07.393 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":37329},{"type":"v1","addr":"172.21.15.97:6833","nonce":37329}]},"active_addr":"172.21.15.97:6833/37329","active_change":"2024-03-14T21:13:42.899797+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2477940144}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1334774064}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2445573642}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":4023220724}]}]} 2024-03-14T21:14:07.397 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-14T21:14:07.397 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-14T21:14:07.397 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.519+0000 7f20c27c3640 1 Processor -- start 2024-03-14T21:14:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.520+0000 7f20c27c3640 1 -- start start 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.520+0000 7f20c27c3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20bc0b3260 0x7f20bc06dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.520+0000 7f20c27c3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc06e6a0 0x7f20bc1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.520+0000 7f20c27c3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20bc0b3740 con 0x7f20bc142cf0 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.520+0000 7f20c27c3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20bc0b4ec0 con 0x7f20bc0b3260 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.520+0000 7f20c27c3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc0b51a0 con 0x7f20bc06e6a0 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20bb7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20bc0b3260 0x7f20bc06dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20c0d60640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc06e6a0 0x7f20bc1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20bbfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f20bc142cf0 0x7f20bc0ba600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47678/0 (socket says 172.21.15.49:47678) 2024-03-14T21:14:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20bbfff640 1 -- 172.21.15.49:0/350735928 learned_addr learned my addr 172.21.15.49:0/350735928 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20baffd640 1 -- 172.21.15.49:0/350735928 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641625479 0 0) 0x7f20bc0b3740 con 0x7f20bc142cf0 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20baffd640 1 -- 172.21.15.49:0/350735928 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a4003520 con 0x7f20bc142cf0 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20c0d60640 1 -- 172.21.15.49:0/350735928 >> v2:172.21.15.97:3300/0 conn(0x7f20bc0b3260 msgr2=0x7f20bc06dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20c0d60640 1 --2- 172.21.15.49:0/350735928 >> v2:172.21.15.97:3300/0 conn(0x7f20bc0b3260 0x7f20bc06dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20c0d60640 1 -- 172.21.15.49:0/350735928 >> v1:172.21.15.97:6789/0 conn(0x7f20bc142cf0 legacy=0x7f20bc0ba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.521+0000 7f20c0d60640 1 -- 172.21.15.49:0/350735928 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc06e130 con 0x7f20bc06e6a0 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20c0d60640 1 --2- 172.21.15.49:0/350735928 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc06e6a0 0x7f20bc1530c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009d90 tx=0x7f20b402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e5ea9583809b85 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20baffd640 1 -- 172.21.15.49:0/350735928 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b402cc50 con 0x7f20bc06e6a0 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20baffd640 1 -- 172.21.15.49:0/350735928 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b402cdb0 con 0x7f20bc06e6a0 2024-03-14T21:14:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20baffd640 1 -- 172.21.15.49:0/350735928 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4034640 con 0x7f20bc06e6a0 2024-03-14T21:14:07.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20c27c3640 1 -- 172.21.15.49:0/350735928 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc06e6a0 msgr2=0x7f20bc1530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20c27c3640 1 --2- 172.21.15.49:0/350735928 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc06e6a0 0x7f20bc1530c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009d90 tx=0x7f20b402bea0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20c27c3640 1 -- 172.21.15.49:0/350735928 shutdown_connections 2024-03-14T21:14:07.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20c27c3640 1 --2- 172.21.15.49:0/350735928 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc06e6a0 0x7f20bc1530c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20c27c3640 1 --2- 172.21.15.49:0/350735928 >> v2:172.21.15.97:3300/0 conn(0x7f20bc0b3260 0x7f20bc06dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.522+0000 7f20c27c3640 1 -- 172.21.15.49:0/350735928 >> 172.21.15.49:0/350735928 conn(0x7f20bc0bad40 msgr2=0x7f20bc0bb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.523+0000 7f20c27c3640 1 -- 172.21.15.49:0/350735928 shutdown_connections 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.523+0000 7f20c27c3640 1 -- 172.21.15.49:0/350735928 wait complete. 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 Processor -- start 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 -- start start 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20bc06e6a0 0x7f20bc13beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc0b3260 0x7f20bc13b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20bc142cf0 0x7f20bc1395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc13a760 con 0x7f20bc142cf0 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc13b120 con 0x7f20bc06e6a0 2024-03-14T21:14:07.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c27c3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc13b400 con 0x7f20bc0b3260 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c0d60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20bc142cf0 0x7f20bc1395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c0d60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20bc142cf0 0x7f20bc1395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37406/0 (socket says 172.21.15.49:37406) 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.524+0000 7f20c0d60640 1 -- 172.21.15.49:0/2798900203 learned_addr learned my addr 172.21.15.49:0/2798900203 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20bbfff640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20bc06e6a0 0x7f20bc13beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20bb7fe640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc0b3260 0x7f20bc13b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c0d60640 1 -- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc0b3260 msgr2=0x7f20bc13b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c0d60640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc0b3260 0x7f20bc13b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c0d60640 1 -- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20bc06e6a0 msgr2=0x7f20bc13beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c0d60640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20bc06e6a0 0x7f20bc13beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c0d60640 1 -- 172.21.15.49:0/2798900203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b4009a70 con 0x7f20bc142cf0 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20bb7fe640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc0b3260 0x7f20bc13b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20bbfff640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20bc06e6a0 0x7f20bc13beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c0d60640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20bc142cf0 0x7f20bc1395b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f20b402c4b0 tx=0x7f20b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20b8ff9640 1 -- 172.21.15.49:0/2798900203 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b403e610 con 0x7f20bc142cf0 2024-03-14T21:14:07.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20b8ff9640 1 -- 172.21.15.49:0/2798900203 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b403e770 con 0x7f20bc142cf0 2024-03-14T21:14:07.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20b8ff9640 1 -- 172.21.15.49:0/2798900203 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b403eb20 con 0x7f20bc142cf0 2024-03-14T21:14:07.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20bc0a3e50 con 0x7f20bc142cf0 2024-03-14T21:14:07.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.525+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20bc141a30 con 0x7f20bc142cf0 2024-03-14T21:14:07.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.526+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20bc0b43f0 con 0x7f20bc142cf0 2024-03-14T21:14:07.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.526+0000 7f20b8ff9640 1 -- 172.21.15.49:0/2798900203 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f20b403b050 con 0x7f20bc142cf0 2024-03-14T21:14:07.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.527+0000 7f20b8ff9640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f20a806a7a0 0x7f20a806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.527+0000 7f20b8ff9640 1 -- 172.21.15.49:0/2798900203 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20b40aae50 con 0x7f20bc142cf0 2024-03-14T21:14:07.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.527+0000 7f20bbfff640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f20a806a7a0 0x7f20a806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.528+0000 7f20bbfff640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f20a806a7a0 0x7f20a806cc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007fd0 tx=0x7f20b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.531+0000 7f20b8ff9640 1 -- 172.21.15.49:0/2798900203 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f20b4076420 con 0x7f20bc142cf0 2024-03-14T21:14:07.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.711+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20bc13b540 con 0x7f20bc142cf0 2024-03-14T21:14:07.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.712+0000 7f20b8ff9640 1 -- 172.21.15.49:0/2798900203 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f20b4009450 con 0x7f20bc142cf0 2024-03-14T21:14:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:07.696 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:04.014218+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f20a806a7a0 msgr2=0x7f20a806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f20a806a7a0 0x7f20a806cc60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007fd0 tx=0x7f20b0009040 comp rx=0 tx=0).stop 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20bc142cf0 msgr2=0x7f20bc1395b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20bc142cf0 0x7f20bc1395b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f20b402c4b0 tx=0x7f20b4004a40 comp rx=0 tx=0).stop 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 shutdown_connections 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f20a806a7a0 0x7f20a806cc60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20bc142cf0 0x7f20bc1395b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20bc0b3260 0x7f20bc13b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 --2- 172.21.15.49:0/2798900203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20bc06e6a0 0x7f20bc13beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 >> 172.21.15.49:0/2798900203 conn(0x7f20bc0bad40 msgr2=0x7f20bc138190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:07.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.714+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 shutdown_connections 2024-03-14T21:14:07.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.715+0000 7f20c27c3640 1 -- 172.21.15.49:0/2798900203 wait complete. 2024-03-14T21:14:07.708 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-14T21:14:07.708 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:07.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.831+0000 7fe1e65e7640 1 Processor -- start 2024-03-14T21:14:07.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.831+0000 7fe1e65e7640 1 -- start start 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1e65e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1e00baef0 0x7fe1e01480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1e65e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e006b620 0x7fe1e00bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1e65e7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e006a6e0 con 0x7fe1e00bc940 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1e65e7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e006ad10 con 0x7fe1e00baef0 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1e65e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00bc2b0 con 0x7fe1e006b620 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1df7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e006b620 0x7fe1e00bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1e4b84640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe1e00bc940 0x7fe1e014e3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47694/0 (socket says 172.21.15.49:47694) 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1e4b84640 1 -- 172.21.15.49:0/4106052293 learned_addr learned my addr 172.21.15.49:0/4106052293 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.832+0000 7fe1dffff640 1 --2- 172.21.15.49:0/4106052293 >> v2:172.21.15.97:3300/0 conn(0x7fe1e00baef0 0x7fe1e01480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.833+0000 7fe1df7fe640 1 -- 172.21.15.49:0/4106052293 >> v2:172.21.15.97:3300/0 conn(0x7fe1e00baef0 msgr2=0x7fe1e01480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:07.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.833+0000 7fe1df7fe640 1 --2- 172.21.15.49:0/4106052293 >> v2:172.21.15.97:3300/0 conn(0x7fe1e00baef0 0x7fe1e01480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.833+0000 7fe1df7fe640 1 -- 172.21.15.49:0/4106052293 >> v1:172.21.15.97:6789/0 conn(0x7fe1e00bc940 legacy=0x7fe1e014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.833+0000 7fe1df7fe640 1 -- 172.21.15.49:0/4106052293 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e006bab0 con 0x7fe1e006b620 2024-03-14T21:14:07.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.833+0000 7fe1df7fe640 1 --2- 172.21.15.49:0/4106052293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e006b620 0x7fe1e00bf7f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709a7fc49c0c380c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.833+0000 7fe1deffd640 1 -- 172.21.15.49:0/4106052293 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d002e070 con 0x7fe1e006b620 2024-03-14T21:14:07.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1deffd640 1 -- 172.21.15.49:0/4106052293 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d00315a0 con 0x7fe1e006b620 2024-03-14T21:14:07.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1deffd640 1 -- 172.21.15.49:0/4106052293 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0031930 con 0x7fe1e006b620 2024-03-14T21:14:07.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 -- 172.21.15.49:0/4106052293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e006b620 msgr2=0x7fe1e00bf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/4106052293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e006b620 0x7fe1e00bf7f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002b840 comp rx=0 tx=0).stop 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 -- 172.21.15.49:0/4106052293 shutdown_connections 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/4106052293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e006b620 0x7fe1e00bf7f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/4106052293 >> v2:172.21.15.97:3300/0 conn(0x7fe1e00baef0 0x7fe1e01480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 -- 172.21.15.49:0/4106052293 >> 172.21.15.49:0/4106052293 conn(0x7fe1e00bcd80 msgr2=0x7fe1e00bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 -- 172.21.15.49:0/4106052293 shutdown_connections 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.834+0000 7fe1e65e7640 1 -- 172.21.15.49:0/4106052293 wait complete. 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.835+0000 7fe1e65e7640 1 Processor -- start 2024-03-14T21:14:07.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.835+0000 7fe1e65e7640 1 -- start start 2024-03-14T21:14:07.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1e65e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00baef0 0x7fe1e0146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1e65e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1e00bc940 0x7fe1e00a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1e65e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e01e0670 0x7fe1e01e3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1e65e7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e0146a90 con 0x7fe1e00baef0 2024-03-14T21:14:07.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1e65e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01e01e0 con 0x7fe1e00bc940 2024-03-14T21:14:07.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1e65e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00a2d20 con 0x7fe1e01e0670 2024-03-14T21:14:07.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00baef0 0x7fe1e0146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00baef0 0x7fe1e0146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37418/0 (socket says 172.21.15.49:37418) 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 -- 172.21.15.49:0/3725145643 learned_addr learned my addr 172.21.15.49:0/3725145643 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1df7fe640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1e00bc940 0x7fe1e00a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1e4b84640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e01e0670 0x7fe1e01e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 -- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e01e0670 msgr2=0x7fe1e01e3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e01e0670 0x7fe1e01e3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 -- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1e00bc940 msgr2=0x7fe1e00a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1e00bc940 0x7fe1e00a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.836+0000 7fe1dffff640 1 -- 172.21.15.49:0/3725145643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0009400 con 0x7fe1e00baef0 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.837+0000 7fe1dffff640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00baef0 0x7fe1e0146550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400e9b0 tx=0x7fe1d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.837+0000 7fe1dcff9640 1 -- 172.21.15.49:0/3725145643 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4009670 con 0x7fe1e00baef0 2024-03-14T21:14:07.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.837+0000 7fe1dcff9640 1 -- 172.21.15.49:0/3725145643 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d400c9a0 con 0x7fe1e00baef0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.837+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e01dfeb0 con 0x7fe1e00baef0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.837+0000 7fe1dcff9640 1 -- 172.21.15.49:0/3725145643 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d400cd50 con 0x7fe1e00baef0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.837+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e014ee10 con 0x7fe1e00baef0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.838+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e0066bb0 con 0x7fe1e00baef0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.838+0000 7fe1dcff9640 1 -- 172.21.15.49:0/3725145643 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe1d4013470 con 0x7fe1e00baef0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.839+0000 7fe1dcff9640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe1cc06a7d0 0x7fe1cc06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.839+0000 7fe1dcff9640 1 -- 172.21.15.49:0/3725145643 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1d4088ee0 con 0x7fe1e00baef0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.839+0000 7fe1df7fe640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe1cc06a7d0 0x7fe1cc06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:07.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.839+0000 7fe1df7fe640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe1cc06a7d0 0x7fe1cc06cc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0036000 tx=0x7fe1d003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:07.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:07.843+0000 7fe1dcff9640 1 -- 172.21.15.49:0/3725145643 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe1d40543b0 con 0x7fe1e00baef0 2024-03-14T21:14:08.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.023+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1e00ba320 con 0x7fe1e00baef0 2024-03-14T21:14:08.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.024+0000 7fe1dcff9640 1 -- 172.21.15.49:0/3725145643 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fe1d40579d0 con 0x7fe1e00baef0 2024-03-14T21:14:08.008 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:08.008 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:04.014218+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:08.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe1cc06a7d0 msgr2=0x7fe1cc06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe1cc06a7d0 0x7fe1cc06cc90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0036000 tx=0x7fe1d003d040 comp rx=0 tx=0).stop 2024-03-14T21:14:08.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00baef0 msgr2=0x7fe1e0146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00baef0 0x7fe1e0146550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400e9b0 tx=0x7fe1d400ee70 comp rx=0 tx=0).stop 2024-03-14T21:14:08.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 shutdown_connections 2024-03-14T21:14:08.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe1cc06a7d0 0x7fe1cc06cc90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1e01e0670 0x7fe1e01e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1e00bc940 0x7fe1e00a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 --2- 172.21.15.49:0/3725145643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00baef0 0x7fe1e0146550 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 >> 172.21.15.49:0/3725145643 conn(0x7fe1e00bcd80 msgr2=0x7fe1e0072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 shutdown_connections 2024-03-14T21:14:08.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.027+0000 7fe1e65e7640 1 -- 172.21.15.49:0/3725145643 wait complete. 2024-03-14T21:14:08.021 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-14T21:14:08.021 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-14T21:14:08.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-14T21:14:08.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-14T21:14:08.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-14T21:14:08.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-14T21:14:08.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-14T21:14:08.024 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-14T21:14:08.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.222+0000 7ff48ae73640 1 Processor -- start 2024-03-14T21:14:08.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.222+0000 7ff48ae73640 1 -- start start 2024-03-14T21:14:08.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff48ae73640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4840b8980 0x7ff484143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff48ae73640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840b18c0 con 0x7ff484066bb0 2024-03-14T21:14:08.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff48ae73640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840bd190 con 0x7ff4840a0f30 2024-03-14T21:14:08.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff48ae73640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840b1d50 con 0x7ff4840b8980 2024-03-14T21:14:08.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ffb4e985640 1 Processor -- start 2024-03-14T21:14:08.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7fb657619640 1 Processor -- start 2024-03-14T21:14:08.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7f81effff640 1 Processor -- start 2024-03-14T21:14:08.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7f81effff640 1 -- start start 2024-03-14T21:14:08.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff489670640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4840b8980 0x7ff484143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff489670640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4840b8980 0x7ff484143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46316/0 (socket says 172.21.15.49:46316) 2024-03-14T21:14:08.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff489670640 1 -- 172.21.15.49:0/2856165994 learned_addr learned my addr 172.21.15.49:0/2856165994 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ff48a672640 1 --1- 172.21.15.49:0/2856165994 >> v1:172.21.15.97:6789/0 conn(0x7ff4840a0f30 0x7ff48414b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47706/0 (socket says 172.21.15.49:47706) 2024-03-14T21:14:08.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f468b106640 1 Processor -- start 2024-03-14T21:14:08.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7ffb4e985640 1 -- start start 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81effff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f0069f50 0x7f81f006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81effff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81f00b8ca0 0x7f81f00b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81effff640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f0071f50 con 0x7f81f0152370 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81effff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f0070e10 con 0x7f81f00b8ca0 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81effff640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00b9aa0 con 0x7f81f0069f50 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81ee7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81f00b8ca0 0x7f81f00b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81ee7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f81f00b8ca0 0x7f81f00b9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37420/0 (socket says 172.21.15.49:37420) 2024-03-14T21:14:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f81ee7fc640 1 -- 172.21.15.49:0/1735635763 learned_addr learned my addr 172.21.15.49:0/1735635763 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7ffb4e985640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffb4000b3e0 0x7ffb400ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7ffb4e985640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4003de10 con 0x7ffb40042e50 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7ffb4e985640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4003e6b0 con 0x7ffb40008e60 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7ffb4e985640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4003e060 con 0x7ffb4000b3e0 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7ffb4d182640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffb4000b3e0 0x7ffb400ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7ffb4d182640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffb4000b3e0 0x7ffb400ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37448/0 (socket says 172.21.15.49:37448) 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7ffb4d983640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffb40008e60 0x7ffb400cb1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47732/0 (socket says 172.21.15.49:47732) 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7fda3f59e640 1 Processor -- start 2024-03-14T21:14:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7fda3f59e640 1 -- start start 2024-03-14T21:14:08.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7ff489670640 1 -- 172.21.15.49:0/2856165994 >> v1:172.21.15.97:6789/0 conn(0x7ff4840a0f30 legacy=0x7ff48414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7ff489670640 1 -- 172.21.15.49:0/2856165994 >> v1:172.21.15.49:6789/0 conn(0x7ff484066bb0 legacy=0x7ff4840bca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7ff489670640 1 -- 172.21.15.49:0/2856165994 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840b7b10 con 0x7ff4840b8980 2024-03-14T21:14:08.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7ffb4d983640 1 -- 172.21.15.49:0/2378474969 learned_addr learned my addr 172.21.15.49:0/2378474969 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fda3f59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda400b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fda3f59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda400b1240 0x7fda4014a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fda3f59e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda400b16d0 con 0x7fda4006d200 2024-03-14T21:14:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fda3f59e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4014a6e0 con 0x7fda400b1240 2024-03-14T21:14:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fda3f59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda400b7030 con 0x7fda400b7df0 2024-03-14T21:14:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fa589385640 1 Processor -- start 2024-03-14T21:14:08.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7f468b106640 1 -- start start 2024-03-14T21:14:08.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7ff489670640 1 --2- 172.21.15.49:0/2856165994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4840b8980 0x7ff484143ef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff48002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3744fce648282c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fa589385640 1 -- start start 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.224+0000 7fb657619640 1 -- start start 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.225+0000 7fb657619640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb648040310 0x7fb64803c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb657619640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb64803ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb657619640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6480475a0 con 0x7fb6480d4480 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb657619640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6480cb720 con 0x7fb648040310 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb657619640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64803d3c0 con 0x7fb64802c260 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb654bb4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb64803ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb6553b5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb648040310 0x7fb64803c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb654bb4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb64803ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46328/0 (socket says 172.21.15.49:46328) 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb6553b5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb648040310 0x7fb64803c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37434/0 (socket says 172.21.15.49:37434) 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb654bb4640 1 -- 172.21.15.49:0/2328133201 learned_addr learned my addr 172.21.15.49:0/2328133201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb6553b5640 1 -- 172.21.15.49:0/2328133201 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 msgr2=0x7fb64803ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda400b1240 0x7fda4014a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7ff488e6f640 1 -- 172.21.15.49:0/2856165994 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48002f840 con 0x7ff4840b8980 2024-03-14T21:14:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f468b106640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 0x7f467c041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f468b106640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f467c00b210 0x7f467c042010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f468b106640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c03f650 con 0x7f467c03fae0 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f468b106640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c03ed40 con 0x7f467c00b210 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f468b106640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c03f000 con 0x7f467c008e90 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4683fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f467c00b210 0x7f467c042010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4683fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f467c00b210 0x7f467c042010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33732/0 (socket says 172.21.15.49:33732) 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4683fff640 1 -- 172.21.15.49:0/1165657154 learned_addr learned my addr 172.21.15.49:0/1165657154 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4688ea2640 1 --2- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 0x7f467c041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3ed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda400b1240 0x7fda4014a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33726/0 (socket says 172.21.15.49:33726) 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3ed9d640 1 -- 172.21.15.49:0/3723239191 learned_addr learned my addr 172.21.15.49:0/3723239191 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3e59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fda4006d200 0x7fda40146a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36872/0 (socket says 172.21.15.49:36872) 2024-03-14T21:14:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7ff488e6f640 1 -- 172.21.15.49:0/2856165994 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48002f9a0 con 0x7ff4840b8980 2024-03-14T21:14:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4683fff640 1 -- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 msgr2=0x7f467c041ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4683fff640 1 --2- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 0x7f467c041ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4683fff640 1 -- 172.21.15.49:0/1165657154 >> v1:172.21.15.97:6789/0 conn(0x7f467c03fae0 legacy=0x7f467c0d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7f4683fff640 1 -- 172.21.15.49:0/1165657154 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c03f2e0 con 0x7f467c00b210 2024-03-14T21:14:08.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7fb6553b5640 1 --2- 172.21.15.49:0/2328133201 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb64803ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fb6553b5640 1 -- 172.21.15.49:0/2328133201 >> v1:172.21.15.97:6789/0 conn(0x7fb6480d4480 legacy=0x7fb648046e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fb6553b5640 1 -- 172.21.15.49:0/2328133201 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64803b760 con 0x7fb648040310 2024-03-14T21:14:08.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7ff488e6f640 1 -- 172.21.15.49:0/2856165994 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48002fd70 con 0x7ff4840b8980 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646038148 0 0) 0x7ffb4003e6b0 con 0x7ffb40008e60 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb34003520 con 0x7ffb40008e60 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166353759 0 0) 0x7ffb4003de10 con 0x7ffb40042e50 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb4003e6b0 con 0x7ffb40042e50 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f81ee7fc640 1 -- 172.21.15.49:0/1735635763 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f0069f50 msgr2=0x7f81f006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f81ee7fc640 1 --2- 172.21.15.49:0/1735635763 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f0069f50 0x7f81f006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f81ee7fc640 1 -- 172.21.15.49:0/1735635763 >> v1:172.21.15.49:6789/0 conn(0x7f81f0152370 legacy=0x7f81f014bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.226+0000 7f81ee7fc640 1 -- 172.21.15.49:0/1735635763 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f00b8070 con 0x7f81f00b8ca0 2024-03-14T21:14:08.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3dd9b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda400b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3ed9d640 1 -- 172.21.15.49:0/3723239191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 msgr2=0x7fda400b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3ed9d640 1 --2- 172.21.15.49:0/3723239191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda400b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3ed9d640 1 -- 172.21.15.49:0/3723239191 >> v1:172.21.15.49:6789/0 conn(0x7fda4006d200 legacy=0x7fda40146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.227+0000 7fda3ed9d640 1 -- 172.21.15.49:0/3723239191 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400b7310 con 0x7fda400b1240 2024-03-14T21:14:08.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7f7f3d362640 1 Processor -- start 2024-03-14T21:14:08.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa589385640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c03cd00 0x7fa57c0411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa589385640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa57c0d9530 0x7fa57c0e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa589385640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c03dc70 con 0x7fa57c041720 2024-03-14T21:14:08.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa589385640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57c02bea0 con 0x7fa57c0d9530 2024-03-14T21:14:08.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa589385640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c03d420 con 0x7fa57c03cd00 2024-03-14T21:14:08.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1465655135 0 0) 0x7ffb4003e6b0 con 0x7ffb40042e50 2024-03-14T21:14:08.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 >> v2:172.21.15.49:3300/0 conn(0x7ffb4000b3e0 msgr2=0x7ffb400ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 --2- 172.21.15.49:0/2378474969 >> v2:172.21.15.49:3300/0 conn(0x7ffb4000b3e0 0x7ffb400ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 >> v1:172.21.15.97:6789/0 conn(0x7ffb40008e60 legacy=0x7ffb400cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa583fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c03cd00 0x7fa57c0411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa583fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c03cd00 0x7fa57c0411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46362/0 (socket says 172.21.15.49:46362) 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4003e340 con 0x7ffb40042e50 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7ffb4804e550 con 0x7ffb40042e50 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa583fff640 1 -- 172.21.15.49:0/1903558721 learned_addr learned my addr 172.21.15.49:0/1903558721 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 >> v1:172.21.15.49:6789/0 conn(0x7ffb40042e50 legacy=0x7ffb400d8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb34004b60 0x7ffb34004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34003760 0x7ffb34003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb34004120 0x7ffb3400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4003e060 con 0x7ffb34003760 2024-03-14T21:14:08.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34005950 con 0x7ffb34004120 2024-03-14T21:14:08.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34005ac0 con 0x7ffb34004b60 2024-03-14T21:14:08.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ff48ae73640 1 -- 172.21.15.49:0/2856165994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4840b8980 msgr2=0x7ff484143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ff48ae73640 1 --2- 172.21.15.49:0/2856165994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4840b8980 0x7ff484143ef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff48002ba20 comp rx=0 tx=0).stop 2024-03-14T21:14:08.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7f81ee7fc640 1 --2- 172.21.15.49:0/1735635763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f00b8ca0 0x7f81f00b9560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81e800b1d0 tx=0x7f81e802c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccddf002ca11d70 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4d182640 1 --2- 172.21.15.49:0/2378474969 >> v2:172.21.15.49:3300/0 conn(0x7ffb4000b3e0 0x7ffb400ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:08.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa588b84640 1 --2- 172.21.15.49:0/1903558721 >> v2:172.21.15.97:3300/0 conn(0x7fa57c0d9530 0x7fa57c0e00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7f7f3d362640 1 -- start start 2024-03-14T21:14:08.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4e184640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb34004120 0x7ffb3400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ffb4d983640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb34004b60 0x7ffb34004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ff48ae73640 1 -- 172.21.15.49:0/2856165994 shutdown_connections 2024-03-14T21:14:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ff48ae73640 1 --2- 172.21.15.49:0/2856165994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4840b8980 0x7ff484143ef0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7ff48ae73640 1 -- 172.21.15.49:0/2856165994 >> 172.21.15.49:0/2856165994 conn(0x7ff48406a370 msgr2=0x7ff4840bc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7f4683fff640 1 --2- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c00b210 0x7f467c042010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46840687c0 tx=0x7f46840a5530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45e83a700b45012 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa583fff640 1 -- 172.21.15.49:0/1903558721 >> v2:172.21.15.97:3300/0 conn(0x7fa57c0d9530 msgr2=0x7fa57c0e00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa583fff640 1 --2- 172.21.15.49:0/1903558721 >> v2:172.21.15.97:3300/0 conn(0x7fa57c0d9530 0x7fa57c0e00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa583fff640 1 -- 172.21.15.49:0/1903558721 >> v1:172.21.15.49:6789/0 conn(0x7fa57c041720 legacy=0x7fa57c0d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7fa583fff640 1 -- 172.21.15.49:0/1903558721 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c041b80 con 0x7fa57c03cd00 2024-03-14T21:14:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f46837fe640 1 -- 172.21.15.49:0/1165657154 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840aa7d0 con 0x7f467c00b210 2024-03-14T21:14:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f46837fe640 1 -- 172.21.15.49:0/1165657154 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840a8050 con 0x7f467c00b210 2024-03-14T21:14:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7fb6553b5640 1 --2- 172.21.15.49:0/2328133201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb648040310 0x7fb64803c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0097d0 tx=0x7fb64c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492c85696f24dea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ff48ae73640 1 -- 172.21.15.49:0/2856165994 shutdown_connections 2024-03-14T21:14:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ff48ae73640 1 -- 172.21.15.49:0/2856165994 wait complete. 2024-03-14T21:14:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ffb4d182640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34003760 0x7ffb34003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f46837fe640 1 -- 172.21.15.49:0/1165657154 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840aac30 con 0x7f467c00b210 2024-03-14T21:14:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ffb4e184640 1 -- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb34004b60 msgr2=0x7ffb34004f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ffb4e184640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb34004b60 0x7ffb34004f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ffb4e184640 1 -- 172.21.15.49:0/2378474969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34003760 msgr2=0x7ffb34003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ffb4e184640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34003760 0x7ffb34003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7ffb4e184640 1 -- 172.21.15.49:0/2378474969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4003e340 con 0x7ffb34004120 2024-03-14T21:14:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7f81edffb640 1 -- 172.21.15.49:0/1735635763 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e802cae0 con 0x7f81f00b8ca0 2024-03-14T21:14:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7f81edffb640 1 -- 172.21.15.49:0/1735635763 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e802cc40 con 0x7f81f00b8ca0 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.228+0000 7f81edffb640 1 -- 172.21.15.49:0/1735635763 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80343f0 con 0x7f81f00b8ca0 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f7f3d362640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38146b00 0x7f7f38146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f7f3d362640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3806cbd0 con 0x7f7f38071bc0 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f7f3d362640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f381474f0 con 0x7f7f380b9630 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f7f3d362640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b2ad0 con 0x7f7f38146b00 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f7f37fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7f38071bc0 0x7f7f38140980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36888/0 (socket says 172.21.15.49:36888) 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f7f377fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7f380b9630 0x7f7f380b1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47758/0 (socket says 172.21.15.49:47758) 2024-03-14T21:14:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f7f37fff640 1 -- 172.21.15.49:0/2165538883 learned_addr learned my addr 172.21.15.49:0/2165538883 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:08.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7fda3ed9d640 1 --2- 172.21.15.49:0/3723239191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 0x7fda4014a1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ed0 tx=0x7fda380302d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6642d23dc10eea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f81effff640 1 -- 172.21.15.49:0/1735635763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f00b8ca0 msgr2=0x7f81f00b9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7f81effff640 1 --2- 172.21.15.49:0/1735635763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f00b8ca0 0x7f81f00b9560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81e800b1d0 tx=0x7f81e802c140 comp rx=0 tx=0).stop 2024-03-14T21:14:08.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f3cb61640 1 --2- 172.21.15.49:0/2165538883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38146b00 0x7f7f38146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 -- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c00b210 msgr2=0x7f467c042010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f36ffd640 1 -- 172.21.15.49:0/2165538883 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156971752 0 0) 0x7f7f3806cbd0 con 0x7f7f38071bc0 2024-03-14T21:14:08.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f36ffd640 1 -- 172.21.15.49:0/2165538883 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f20003520 con 0x7f7f38071bc0 2024-03-14T21:14:08.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 --2- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c00b210 0x7f467c042010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46840687c0 tx=0x7f46840a5530 comp rx=0 tx=0).stop 2024-03-14T21:14:08.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7ffb4e184640 1 -- 172.21.15.49:0/2378474969 reap_dead start 2024-03-14T21:14:08.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fa583fff640 1 --2- 172.21.15.49:0/1903558721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c03cd00 0x7fa57c0411e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5840687c0 tx=0x7fa5840a55f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e0ee7a211016a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f81effff640 1 -- 172.21.15.49:0/1735635763 shutdown_connections 2024-03-14T21:14:08.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f81effff640 1 --2- 172.21.15.49:0/1735635763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f00b8ca0 0x7f81f00b9560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f81effff640 1 --2- 172.21.15.49:0/1735635763 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f0069f50 0x7f81f006ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f81effff640 1 -- 172.21.15.49:0/1735635763 >> 172.21.15.49:0/1735635763 conn(0x7f81f006adf0 msgr2=0x7f81f006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7fb647fff640 1 -- 172.21.15.49:0/2328133201 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c02d040 con 0x7fb648040310 2024-03-14T21:14:08.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.229+0000 7fb647fff640 1 -- 172.21.15.49:0/2328133201 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c031810 con 0x7fb648040310 2024-03-14T21:14:08.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f36ffd640 1 -- 172.21.15.49:0/2165538883 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678419277 0 0) 0x7f7f381474f0 con 0x7f7f380b9630 2024-03-14T21:14:08.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f36ffd640 1 -- 172.21.15.49:0/2165538883 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f3806cbd0 con 0x7f7f380b9630 2024-03-14T21:14:08.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f81effff640 1 -- 172.21.15.49:0/1735635763 shutdown_connections 2024-03-14T21:14:08.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f81effff640 1 -- 172.21.15.49:0/1735635763 wait complete. 2024-03-14T21:14:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 -- 172.21.15.49:0/1165657154 shutdown_connections 2024-03-14T21:14:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 --2- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c00b210 0x7f467c042010 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 --2- 172.21.15.49:0/1165657154 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 0x7f467c041ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 -- 172.21.15.49:0/1165657154 >> 172.21.15.49:0/1165657154 conn(0x7f467c00a590 msgr2=0x7f467c00a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 -- 172.21.15.49:0/1165657154 shutdown_connections 2024-03-14T21:14:08.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f468b106640 1 -- 172.21.15.49:0/1165657154 wait complete. 2024-03-14T21:14:08.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f3cb61640 1 -- 172.21.15.49:0/2165538883 >> v1:172.21.15.97:6789/0 conn(0x7f7f380b9630 legacy=0x7f7f380b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f3cb61640 1 -- 172.21.15.49:0/2165538883 >> v1:172.21.15.49:6789/0 conn(0x7f7f38071bc0 legacy=0x7f7f38140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fda3d59a640 1 -- 172.21.15.49:0/3723239191 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38030e70 con 0x7fda400b1240 2024-03-14T21:14:08.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fda3d59a640 1 -- 172.21.15.49:0/3723239191 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda380383f0 con 0x7fda400b1240 2024-03-14T21:14:08.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7ff48ae73640 1 Processor -- start 2024-03-14T21:14:08.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fda3d59a640 1 -- 172.21.15.49:0/3723239191 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38038780 con 0x7fda400b1240 2024-03-14T21:14:08.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fa582ffd640 1 -- 172.21.15.49:0/1903558721 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840aa7d0 con 0x7fa57c03cd00 2024-03-14T21:14:08.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fa582ffd640 1 -- 172.21.15.49:0/1903558721 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5840a8050 con 0x7fa57c03cd00 2024-03-14T21:14:08.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fa582ffd640 1 -- 172.21.15.49:0/1903558721 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840aac30 con 0x7fa57c03cd00 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7f7f3cb61640 1 -- 172.21.15.49:0/2165538883 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3806c860 con 0x7f7f38146b00 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fb657619640 1 -- 172.21.15.49:0/2328133201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb648040310 msgr2=0x7fb64803c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7fb657619640 1 --2- 172.21.15.49:0/2328133201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb648040310 0x7fb64803c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0097d0 tx=0x7fb64c02b8b0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fb657619640 1 -- 172.21.15.49:0/2328133201 shutdown_connections 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fb657619640 1 --2- 172.21.15.49:0/2328133201 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb64803ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fb657619640 1 --2- 172.21.15.49:0/2328133201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb648040310 0x7fb64803c510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fb657619640 1 -- 172.21.15.49:0/2328133201 >> 172.21.15.49:0/2328133201 conn(0x7fb648043ae0 msgr2=0x7fb648046780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7ffb4e184640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb34004120 0x7ffb3400db90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb48068970 tx=0x7ffb340057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41446cbce1a5569 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480a6930 con 0x7ffb34004120 2024-03-14T21:14:08.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb480a6a90 con 0x7ffb34004120 2024-03-14T21:14:08.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ffb4c981640 1 -- 172.21.15.49:0/2378474969 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480a6e40 con 0x7ffb34004120 2024-03-14T21:14:08.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fb657619640 1 -- 172.21.15.49:0/2328133201 shutdown_connections 2024-03-14T21:14:08.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.230+0000 7ff48ae73640 1 -- start start 2024-03-14T21:14:08.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fda3f59e640 1 -- 172.21.15.49:0/3723239191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 msgr2=0x7fda4014a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fda3f59e640 1 --2- 172.21.15.49:0/3723239191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 0x7fda4014a1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ed0 tx=0x7fda380302d0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ff48ae73640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff484066bb0 0x7ff484145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ff48ae73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4840a0f30 0x7ff48413f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ff48ae73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4840b8980 0x7ff48413fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fda3f59e640 1 -- 172.21.15.49:0/3723239191 shutdown_connections 2024-03-14T21:14:08.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fda3f59e640 1 --2- 172.21.15.49:0/3723239191 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda400b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fda3f59e640 1 --2- 172.21.15.49:0/3723239191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 0x7fda4014a1a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7fda3f59e640 1 -- 172.21.15.49:0/3723239191 >> 172.21.15.49:0/3723239191 conn(0x7fda400b8b10 msgr2=0x7fda400b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ff48ae73640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4841475f0 con 0x7ff4840b8980 2024-03-14T21:14:08.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fda3f59e640 1 -- 172.21.15.49:0/3723239191 shutdown_connections 2024-03-14T21:14:08.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ff48ae73640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff484140100 con 0x7ff4840a0f30 2024-03-14T21:14:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ffb4e985640 1 -- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb34004120 msgr2=0x7ffb3400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ffb4e985640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb34004120 0x7ffb3400db90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffb48068970 tx=0x7ffb340057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fa589385640 1 -- 172.21.15.49:0/1903558721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c03cd00 msgr2=0x7fa57c0411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fa589385640 1 --2- 172.21.15.49:0/1903558721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c03cd00 0x7fa57c0411e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5840687c0 tx=0x7fa5840a55f0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fda3f59e640 1 -- 172.21.15.49:0/3723239191 wait complete. 2024-03-14T21:14:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.231+0000 7ff48ae73640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48413eca0 con 0x7ff484066bb0 2024-03-14T21:14:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff484066bb0 0x7ff484145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff484066bb0 0x7ff484145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46398/0 (socket says 172.21.15.49:46398) 2024-03-14T21:14:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 -- 172.21.15.49:0/1347882502 learned_addr learned my addr 172.21.15.49:0/1347882502 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489670640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4840a0f30 0x7ff48413f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff48a672640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4840b8980 0x7ff48413fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7f81effff640 1 Processor -- start 2024-03-14T21:14:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7f468b106640 1 Processor -- start 2024-03-14T21:14:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7f468b106640 1 -- start start 2024-03-14T21:14:08.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7f7f3cb61640 1 --2- 172.21.15.49:0/2165538883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38146b00 0x7f7f38146fb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097d0 tx=0x7f7f2c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8dbdab4907af432 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 -- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4840a0f30 msgr2=0x7ff48413f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4840a0f30 0x7ff48413f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 -- 172.21.15.49:0/1347882502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4840b8980 msgr2=0x7ff48413fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4840b8980 0x7ff48413fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 -- 172.21.15.49:0/1347882502 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800094d0 con 0x7ff484066bb0 2024-03-14T21:14:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7f81effff640 1 -- start start 2024-03-14T21:14:08.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fb657619640 1 -- 172.21.15.49:0/2328133201 wait complete. 2024-03-14T21:14:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7ff489e71640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff484066bb0 0x7ff484145780 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00c7c0 tx=0x7ff47c00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fa589385640 1 -- 172.21.15.49:0/1903558721 shutdown_connections 2024-03-14T21:14:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fa589385640 1 --2- 172.21.15.49:0/1903558721 >> v2:172.21.15.97:3300/0 conn(0x7fa57c0d9530 0x7fa57c0e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fa589385640 1 --2- 172.21.15.49:0/1903558721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c03cd00 0x7fa57c0411e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.232+0000 7fa589385640 1 -- 172.21.15.49:0/1903558721 >> 172.21.15.49:0/1903558721 conn(0x7fa57c046970 msgr2=0x7fa57c046d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f7f36ffd640 1 -- 172.21.15.49:0/2165538883 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c02d040 con 0x7f7f38146b00 2024-03-14T21:14:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f7f36ffd640 1 -- 172.21.15.49:0/2165538883 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c031810 con 0x7f7f38146b00 2024-03-14T21:14:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00aa50 con 0x7ff484066bb0 2024-03-14T21:14:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7ff48ae73640 1 -- 172.21.15.49:0/1347882502 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4840b3810 con 0x7ff484066bb0 2024-03-14T21:14:08.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f468b106640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 0x7f467c01c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7ff48ae73640 1 -- 172.21.15.49:0/1347882502 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48413e8c0 con 0x7ff484066bb0 2024-03-14T21:14:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f81effff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81f0069f50 0x7f81f00b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f81effff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f0152370 0x7f81f01e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f81effff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f01e7e80 0x7f81f0148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f468b106640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f467c00b210 0x7f467c021d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f468b106640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c03fae0 0x7f467c020a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f468b106640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c01fa70 con 0x7f467c00b210 2024-03-14T21:14:08.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7fa589385640 1 -- 172.21.15.49:0/1903558721 shutdown_connections 2024-03-14T21:14:08.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7fa589385640 1 -- 172.21.15.49:0/1903558721 wait complete. 2024-03-14T21:14:08.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f468b106640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c01e460 con 0x7f467c03fae0 2024-03-14T21:14:08.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f468b106640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0cc8b0 con 0x7f467c008e90 2024-03-14T21:14:08.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f81effff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00a1fd0 con 0x7f81f0152370 2024-03-14T21:14:08.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f81effff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00b7270 con 0x7f81f0069f50 2024-03-14T21:14:08.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f81effff640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00a04c0 con 0x7f81f01e7e80 2024-03-14T21:14:08.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7fda3f59e640 1 Processor -- start 2024-03-14T21:14:08.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f46896a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c03fae0 0x7f467c020a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f46896a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c03fae0 0x7f467c020a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33766/0 (socket says 172.21.15.49:33766) 2024-03-14T21:14:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7f46896a3640 1 -- 172.21.15.49:0/3423617810 learned_addr learned my addr 172.21.15.49:0/3423617810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7fda3f59e640 1 -- start start 2024-03-14T21:14:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f01e7e80 0x7f81f0148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f01e7e80 0x7f81f0148d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46408/0 (socket says 172.21.15.49:46408) 2024-03-14T21:14:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f81ef7fe640 1 -- 172.21.15.49:0/3394650878 learned_addr learned my addr 172.21.15.49:0/3394650878 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f81eeffd640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81f0069f50 0x7f81f00b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f46896a3640 1 -- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 msgr2=0x7f467c01c770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T21:14:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f46896a3640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f467c008e90 0x7f467c01c770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f46896a3640 1 -- 172.21.15.49:0/3423617810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f467c00b210 msgr2=0x7f467c021d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T21:14:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f7f3d362640 1 -- 172.21.15.49:0/2165538883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38146b00 msgr2=0x7f7f38146fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f7f3d362640 1 --2- 172.21.15.49:0/2165538883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38146b00 0x7f7f38146fb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097d0 tx=0x7f7f2c02b8b0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00abb0 con 0x7ff484066bb0 2024-03-14T21:14:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f81ee7fc640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f0152370 0x7f81f01e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f4683fff640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f467c00b210 0x7f467c021d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f46896a3640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f467c00b210 0x7f467c021d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f46896a3640 1 -- 172.21.15.49:0/3423617810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468406be70 con 0x7f467c03fae0 2024-03-14T21:14:08.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fa589385640 1 Processor -- start 2024-03-14T21:14:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c012420 con 0x7ff484066bb0 2024-03-14T21:14:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f4683fff640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f467c00b210 0x7f467c021d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fda3f59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda4006d200 0x7fda4013a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fda3f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 0x7fda400a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f46896a3640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c03fae0 0x7f467c020a20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f467000b410 tx=0x7f467000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fda3f59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda401376c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fda3f59e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4013ac10 con 0x7fda4006d200 2024-03-14T21:14:08.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fda3f59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4009f440 con 0x7fda400b1240 2024-03-14T21:14:08.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fda3f59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4009f720 con 0x7fda400b7df0 2024-03-14T21:14:08.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7f81ef7fe640 1 -- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81f0069f50 msgr2=0x7f81f00b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f81ef7fe640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81f0069f50 0x7f81f00b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f81ef7fe640 1 -- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f0152370 msgr2=0x7f81f01e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f81ef7fe640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81f0152370 0x7f81f01e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f81ef7fe640 1 -- 172.21.15.49:0/3394650878 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009b30 con 0x7f81f01e7e80 2024-03-14T21:14:08.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7ff48ae73640 1 -- 172.21.15.49:0/1347882502 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff48413e440 con 0x7ff484066bb0 2024-03-14T21:14:08.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7fb657619640 1 Processor -- start 2024-03-14T21:14:08.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.233+0000 7fb657619640 1 -- start start 2024-03-14T21:14:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f81eeffd640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81f0069f50 0x7f81f00b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fb657619640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb6480cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fb657619640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb648040310 0x7fb648175fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fb657619640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6480d4480 0x7fb6480d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fb657619640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480cbb90 con 0x7fb6480d4480 2024-03-14T21:14:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fb657619640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480d1700 con 0x7fb648040310 2024-03-14T21:14:08.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.234+0000 7fb657619640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480c96e0 con 0x7fb64802c260 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7ffb4e985640 1 -- 172.21.15.49:0/2378474969 shutdown_connections 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7ffb4e985640 1 --2- 172.21.15.49:0/2378474969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb34004120 0x7ffb3400db90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7ffb4e985640 1 -- 172.21.15.49:0/2378474969 >> 172.21.15.49:0/2378474969 conn(0x7ffb4000a560 msgr2=0x7ffb4000a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb655bb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6480d4480 0x7fb6480d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb6553b5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb6480cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb654bb4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb648040310 0x7fb648175fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb655bb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6480d4480 0x7fb6480d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37492/0 (socket says 172.21.15.49:37492) 2024-03-14T21:14:08.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb655bb6640 1 -- 172.21.15.49:0/2406210805 learned_addr learned my addr 172.21.15.49:0/2406210805 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670004020 con 0x7f467c03fae0 2024-03-14T21:14:08.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f7f3d362640 1 -- 172.21.15.49:0/2165538883 shutdown_connections 2024-03-14T21:14:08.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f7f3d362640 1 --2- 172.21.15.49:0/2165538883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38146b00 0x7f7f38146fb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f7f3d362640 1 -- 172.21.15.49:0/2165538883 >> 172.21.15.49:0/2165538883 conn(0x7f7f380b8b10 msgr2=0x7f7f380b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f7f3d362640 1 -- 172.21.15.49:0/2165538883 shutdown_connections 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7ffb4e985640 1 -- 172.21.15.49:0/2378474969 shutdown_connections 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7ffb4e985640 1 -- 172.21.15.49:0/2378474969 wait complete. 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb654bb4640 1 -- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 msgr2=0x7fb6480cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb654bb4640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64802c260 0x7fb6480cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb654bb4640 1 -- 172.21.15.49:0/2406210805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6480d4480 msgr2=0x7fb6480d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb654bb4640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6480d4480 0x7fb6480d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb654bb4640 1 -- 172.21.15.49:0/2406210805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c009400 con 0x7fb648040310 2024-03-14T21:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fb654bb4640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb648040310 0x7fb648175fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6500687c0 tx=0x7fb650073080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff47c012640 con 0x7ff484066bb0 2024-03-14T21:14:08.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7f81ef7fe640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f01e7e80 0x7f81f0148d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b490 tx=0x7f81e400b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fa589385640 1 -- start start 2024-03-14T21:14:08.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa589385640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa57c03cd00 0x7fa57c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fda3ed9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda401376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fda3ed9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda401376c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46424/0 (socket says 172.21.15.49:46424) 2024-03-14T21:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fda3ed9d640 1 -- 172.21.15.49:0/2653200172 learned_addr learned my addr 172.21.15.49:0/2653200172 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fda3dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 0x7fda400a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa589385640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c041720 0x7fa57c0d49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fda3dd9b640 1 -- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 msgr2=0x7fda401376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fda3dd9b640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda400b7df0 0x7fda401376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa583fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa57c03cd00 0x7fa57c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa583fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa57c03cd00 0x7fa57c15f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37496/0 (socket says 172.21.15.49:37496) 2024-03-14T21:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa583fff640 1 -- 172.21.15.49:0/830457315 learned_addr learned my addr 172.21.15.49:0/830457315 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650073960 con 0x7fb648040310 2024-03-14T21:14:08.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650094460 con 0x7fb648040310 2024-03-14T21:14:08.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500947d0 con 0x7fb648040310 2024-03-14T21:14:08.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa589385640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57c0d9530 0x7fa57c16db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7ff47a7fc640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff46806a870 0x7ff46806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7fda3dd9b640 1 -- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda4006d200 msgr2=0x7fda4013a6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T21:14:08.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fda3e59c640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda4006d200 0x7fda4013a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0d4f20 con 0x7fa57c03cd00 2024-03-14T21:14:08.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c00a810 con 0x7fa57c0d9530 2024-03-14T21:14:08.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fda3dd9b640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda4006d200 0x7fda4013a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa588b84640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57c0d9530 0x7fa57c16db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff47c088b40 con 0x7ff484066bb0 2024-03-14T21:14:08.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f468b106640 1 -- 172.21.15.49:0/3423617810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c01ccb0 con 0x7f467c03fae0 2024-03-14T21:14:08.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f468b106640 1 -- 172.21.15.49:0/3423617810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c022260 con 0x7f467c03fae0 2024-03-14T21:14:08.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4670004180 con 0x7f467c03fae0 2024-03-14T21:14:08.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c15e710 con 0x7fa57c041720 2024-03-14T21:14:08.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7fda3dd9b640 1 -- 172.21.15.49:0/2653200172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda38009b30 con 0x7fda400b1240 2024-03-14T21:14:08.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7ff489670640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff46806a870 0x7ff46806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.236+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011830 con 0x7f81f01e7e80 2024-03-14T21:14:08.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e40119b0 con 0x7f81f01e7e80 2024-03-14T21:14:08.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011d60 con 0x7f81f01e7e80 2024-03-14T21:14:08.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7f81effff640 1 -- 172.21.15.49:0/3394650878 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f01e7420 con 0x7f81f01e7e80 2024-03-14T21:14:08.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7f81effff640 1 -- 172.21.15.49:0/3394650878 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f014ee90 con 0x7f81f01e7e80 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7ffb4e985640 1 Processor -- start 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7ffb4e985640 1 -- start start 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4e985640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34004120 0x7ffb40038c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4e985640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb4003cf80 0x7ffb4003d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4e985640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb400375d0 0x7ffb40037a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4e985640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40039190 con 0x7ffb34004120 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4e985640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4003d900 con 0x7ffb400375d0 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4e985640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40037f90 con 0x7ffb4003cf80 2024-03-14T21:14:08.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6480cd240 con 0x7fb648040310 2024-03-14T21:14:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6480c9400 con 0x7fb648040310 2024-03-14T21:14:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fda3dd9b640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 0x7fda400a1980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda3400d520 tx=0x7fda3400d9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda340125f0 con 0x7fda400b1240 2024-03-14T21:14:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda34012750 con 0x7fda400b1240 2024-03-14T21:14:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34012ae0 con 0x7fda400b1240 2024-03-14T21:14:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4009fa00 con 0x7fda400b1240 2024-03-14T21:14:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4006faa0 con 0x7fda400b1240 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fa583fff640 1 -- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c041720 msgr2=0x7fa57c0d49e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fa583fff640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa57c041720 0x7fa57c0d49e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fa583fff640 1 -- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57c0d9530 msgr2=0x7fa57c16db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fa583fff640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57c0d9530 0x7fa57c16db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fa583fff640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58406be70 con 0x7fa57c03cd00 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7fa588b84640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa57c0d9530 0x7fa57c16db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fa583fff640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa57c03cd00 0x7fa57c15f6a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa584063b00 tx=0x7fa584068060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840b68e0 con 0x7fa57c03cd00 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5840b6a40 con 0x7fa57c03cd00 2024-03-14T21:14:08.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7ff47a7fc640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7ff468072920 0x7ff468074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 --> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4680753f0 con 0x7ff468072920 2024-03-14T21:14:08.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff47c08d240 con 0x7ff484066bb0 2024-03-14T21:14:08.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ff489670640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff46806a870 0x7ff46806cd30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff480036000 tx=0x7ff480008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670011630 con 0x7f467c03fae0 2024-03-14T21:14:08.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4670011850 con 0x7f467c03fae0 2024-03-14T21:14:08.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.235+0000 7f7f3d362640 1 -- 172.21.15.49:0/2165538883 wait complete. 2024-03-14T21:14:08.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7f7f3d362640 1 Processor -- start 2024-03-14T21:14:08.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840b6df0 con 0x7fa57c03cd00 2024-03-14T21:14:08.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4d182640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb4003cf80 0x7ffb4003d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4d182640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb4003cf80 0x7ffb4003d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46444/0 (socket says 172.21.15.49:46444) 2024-03-14T21:14:08.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4d182640 1 -- 172.21.15.49:0/2136379896 learned_addr learned my addr 172.21.15.49:0/2136379896 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4d983640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34004120 0x7ffb40038c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ffb4e184640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb400375d0 0x7ffb40037a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f467c0d97c0 con 0x7f467c03fae0 2024-03-14T21:14:08.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.237+0000 7f7f3d362640 1 -- start start 2024-03-14T21:14:08.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f7f3d362640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38071bc0 0x7f7f380a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f7f3d362640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f380b9630 0x7f7f3813f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f7f3d362640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f38146b00 0x7f7f380a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f7f3d362640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380b74c0 con 0x7f7f380b9630 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f7f3d362640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3812c860 con 0x7f7f38146b00 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f7f3d362640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38144270 con 0x7f7f38071bc0 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7f7f377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f380b9630 0x7f7f3813f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f380b9630 0x7f7f3813f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37512/0 (socket says 172.21.15.49:37512) 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 -- 172.21.15.49:0/318410401 learned_addr learned my addr 172.21.15.49:0/318410401 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f46817fa640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f466806a8a0 0x7f466806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb4d983640 1 -- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb4003cf80 msgr2=0x7ffb4003d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4670088a50 con 0x7f467c03fae0 2024-03-14T21:14:08.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda400b78d0 con 0x7fda400b1240 2024-03-14T21:14:08.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f46817fa640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f4668072a40 0x7f4668074e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f4688ea2640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f466806a8a0 0x7f466806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 --> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4668075510 con 0x7f4668072a40 2024-03-14T21:14:08.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c15fd20 con 0x7fa57c03cd00 2024-03-14T21:14:08.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c00ce90 con 0x7fa57c03cd00 2024-03-14T21:14:08.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f4683fff640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f4668072a40 0x7f4668074e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f37fff640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38071bc0 0x7f7f380a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 -- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38071bc0 msgr2=0x7f7f380a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f38071bc0 0x7f7f380a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 -- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f38146b00 msgr2=0x7f7f380a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f38146b00 0x7f7f380a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 -- 172.21.15.49:0/318410401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c009400 con 0x7f7f380b9630 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7f7f377fe640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f380b9630 0x7f7f3813f240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c870 tx=0x7f7f3000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.238+0000 7ff48a672640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7ff468072920 0x7ff468074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb4d983640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb4003cf80 0x7ffb4003d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.240+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4670053e50 con 0x7f467c03fae0 2024-03-14T21:14:08.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.240+0000 7f4683fff640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f4668072a40 0x7f4668074e60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb4d983640 1 -- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb400375d0 msgr2=0x7ffb40037a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb4d983640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb400375d0 0x7ffb40037a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb4d983640 1 -- 172.21.15.49:0/2136379896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40166560 con 0x7ffb34004120 2024-03-14T21:14:08.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb4d983640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34004120 0x7ffb40038c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb44005f80 tx=0x7ffb440017c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44010510 con 0x7ffb34004120 2024-03-14T21:14:08.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb44010670 con 0x7ffb34004120 2024-03-14T21:14:08.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.240+0000 7f4688ea2640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f466806a8a0 0x7f466806cd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46780099d0 tx=0x7f4678009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.239+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb6480406f0 con 0x7fb648040310 2024-03-14T21:14:08.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.240+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb650094930 con 0x7fb648040310 2024-03-14T21:14:08.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f81e401a400 con 0x7f81f01e7e80 2024-03-14T21:14:08.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fda34012c40 con 0x7fda400b1240 2024-03-14T21:14:08.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000a640 con 0x7f7f380b9630 2024-03-14T21:14:08.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.240+0000 7ffb4e985640 1 -- 172.21.15.49:0/2136379896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb400370f0 con 0x7ffb34004120 2024-03-14T21:14:08.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.240+0000 7ffb4e985640 1 -- 172.21.15.49:0/2136379896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb40166280 con 0x7ffb34004120 2024-03-14T21:14:08.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb440109e0 con 0x7ffb34004120 2024-03-14T21:14:08.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.240+0000 7fb645ffb640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb63406a870 0x7fb63406cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6501022a0 con 0x7fb648040310 2024-03-14T21:14:08.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fb645ffb640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7fb634072a90 0x7fb634074eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 --> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb634075560 con 0x7fb634072a90 2024-03-14T21:14:08.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6501048c0 con 0x7fb648040310 2024-03-14T21:14:08.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f380702e0 con 0x7f7f380b9630 2024-03-14T21:14:08.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fb655bb6640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7fb634072a90 0x7fb634074eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f380a10d0 con 0x7f7f380b9630 2024-03-14T21:14:08.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7f81d7fff640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f81dc06a7f0 0x7f81dc06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81f006a330 con 0x7f81f01e7e80 2024-03-14T21:14:08.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81e400e260 con 0x7f81f01e7e80 2024-03-14T21:14:08.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fda2effd640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda2006a870 0x7fda2006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda340895b0 con 0x7fda400b1240 2024-03-14T21:14:08.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fda2effd640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7fda20072a90 0x7fda20074eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 --> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda20075560 con 0x7fda20072a90 2024-03-14T21:14:08.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda34089960 con 0x7fda400b1240 2024-03-14T21:14:08.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== osd.6 v2:172.21.15.97:6800/37902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4668075510 con 0x7f4668072a40 2024-03-14T21:14:08.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.241+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa57c02c490 con 0x7fa57c03cd00 2024-03-14T21:14:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7fb6553b5640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb63406a870 0x7fb63406cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7fda3ed9d640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7fda20072a90 0x7fda20074eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f81eeffd640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f81dc06a7f0 0x7f81dc06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f81d7fff640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7f81dc072990 0x7f81dc074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 --> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81dc075460 con 0x7f81dc072990 2024-03-14T21:14:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7fb655bb6640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7fb634072a90 0x7fb634074eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81e4009190 con 0x7f81f01e7e80 2024-03-14T21:14:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7ffb4e985640 1 -- 172.21.15.49:0/2136379896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb400cb1d0 con 0x7ffb34004120 2024-03-14T21:14:08.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000a7a0 con 0x7f7f380b9630 2024-03-14T21:14:08.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000ab10 con 0x7f7f380b9630 2024-03-14T21:14:08.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f18000fc0 con 0x7f7f380b9630 2024-03-14T21:14:08.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f81ee7fc640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7f81dc072990 0x7f81dc074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7f81eeffd640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f81dc06a7f0 0x7f81dc06ccb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009d60 tx=0x7f81e0009110 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7fda3e59c640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda2006a870 0x7fda2006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.243+0000 7fb6553b5640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb63406a870 0x7fb63406cd30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb64c036000 tx=0x7fb64c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.243+0000 7fda3e59c640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda2006a870 0x7fda2006cd30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda30007fd0 tx=0x7fda30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.242+0000 7ff48a672640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7ff468072920 0x7ff468074d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.243+0000 7f81ee7fc640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7f81dc072990 0x7f81dc074db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.243+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== osd.2 v2:172.21.15.49:6824/37918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb634075560 con 0x7fb634072a90 2024-03-14T21:14:08.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.243+0000 7fda3ed9d640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7fda20072a90 0x7fda20074eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.243+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa5840b3050 con 0x7fa57c03cd00 2024-03-14T21:14:08.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7fa580ff9640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa56806a7d0 0x7fa56806cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5841243b0 con 0x7fa57c03cd00 2024-03-14T21:14:08.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7fa580ff9640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7fa5680729f0 0x7fa568074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5680754c0 con 0x7fa5680729f0 2024-03-14T21:14:08.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5841269d0 con 0x7fa57c03cd00 2024-03-14T21:14:08.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.243+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== osd.5 v2:172.21.15.97:6816/37921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff4680753f0 con 0x7ff468072920 2024-03-14T21:14:08.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== osd.0 v2:172.21.15.49:6816/37908 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f81dc075460 con 0x7f81dc072990 2024-03-14T21:14:08.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7fa588b84640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7fa5680729f0 0x7fa568074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7fa588b84640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7fa5680729f0 0x7fa568074e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== osd.3 v2:172.21.15.49:6808/37909 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fda20075560 con 0x7fda20072a90 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.244+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7f3000ac70 con 0x7f7f380b9630 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7f7f34ff9640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7f2406a7a0 0x7f7f2406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f30088d90 con 0x7f7f380b9630 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7f7f34ff9640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f7f240729c0 0x7f7f24074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 --> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f24075490 con 0x7f7f240729c0 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f30089140 con 0x7f7f380b9630 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7f7f3cb61640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f7f240729c0 0x7f7f24074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ffb44010b80 con 0x7ffb34004120 2024-03-14T21:14:08.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7fa5837fe640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa56806a7d0 0x7fa56806cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.245+0000 7f7f37fff640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7f2406a7a0 0x7f7f2406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7f7f3cb61640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f7f240729c0 0x7f7f24074de0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7ffb3e7fc640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ffb2c06a870 0x7ffb2c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== osd.4 v2:172.21.15.97:6817/37904 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa5680754c0 con 0x7fa5680729f0 2024-03-14T21:14:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb440886e0 con 0x7ffb34004120 2024-03-14T21:14:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7ffb3e7fc640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7ffb2c072920 0x7ffb2c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7f7f37fff640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7f2406a7a0 0x7f7f2406cc60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f280011e0 tx=0x7f7f28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7ffb4d182640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ffb2c06a870 0x7ffb2c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 --> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb2c0753f0 con 0x7ffb2c072920 2024-03-14T21:14:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb4408ad00 con 0x7ffb34004120 2024-03-14T21:14:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.246+0000 7ffb4e184640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7ffb2c072920 0x7ffb2c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.247+0000 7fa5837fe640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa56806a7d0 0x7fa56806cc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa57c047380 tx=0x7fa5780079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.247+0000 7ffb4d182640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ffb2c06a870 0x7ffb2c06cd30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb4015fd10 tx=0x7ffb38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.247+0000 7ffb4e184640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7ffb2c072920 0x7ffb2c074d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.248+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== osd.7 v2:172.21.15.97:6808/37903 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7f24075490 con 0x7f7f240729c0 2024-03-14T21:14:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.248+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== osd.1 v2:172.21.15.49:6800/37907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ffb2c0753f0 con 0x7ffb2c072920 2024-03-14T21:14:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.264+0000 7ff48ae73640 1 -- 172.21.15.49:0/1347882502 --> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff48413ede0 con 0x7ff468072920 2024-03-14T21:14:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.270+0000 7ff47a7fc640 1 -- 172.21.15.49:0/1347882502 <== osd.5 v2:172.21.15.97:6816/37921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff48413ede0 con 0x7ff468072920 2024-03-14T21:14:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.271+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 --> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f81f00718d0 con 0x7f81dc072990 2024-03-14T21:14:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.271+0000 7ff467fff640 1 -- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7ff468072920 msgr2=0x7ff468074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.271+0000 7ff467fff640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7ff468072920 0x7ff468074d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.271+0000 7f81d7fff640 1 -- 172.21.15.49:0/3394650878 <== osd.0 v2:172.21.15.49:6816/37908 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f81dc075460 con 0x7f81dc072990 2024-03-14T21:14:08.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.271+0000 7ff467fff640 1 -- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff46806a870 msgr2=0x7ff46806cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.271+0000 7ff467fff640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ff46806a870 0x7ff46806cd30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff480036000 tx=0x7ff480008030 comp rx=0 tx=0).stop 2024-03-14T21:14:08.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7ff467fff640 1 -- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff484066bb0 msgr2=0x7ff484145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7ff467fff640 1 --2- 172.21.15.49:0/1347882502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff484066bb0 0x7ff484145780 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00c7c0 tx=0x7ff47c00cc80 comp rx=0 tx=0).stop 2024-03-14T21:14:08.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7ff48a672640 1 -- 172.21.15.49:0/1347882502 reap_dead start 2024-03-14T21:14:08.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7f81dc072990 msgr2=0x7f81dc074db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7f81d5ffb640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7f81dc072990 0x7f81dc074db0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f81dc06a7f0 msgr2=0x7f81dc06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7f81d5ffb640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f81dc06a7f0 0x7f81dc06ccb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009d60 tx=0x7f81e0009110 comp rx=0 tx=0).stop 2024-03-14T21:14:08.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f01e7e80 msgr2=0x7f81f0148d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7f81d5ffb640 1 --2- 172.21.15.49:0/3394650878 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81f01e7e80 0x7f81f0148d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b490 tx=0x7f81e400b890 comp rx=0 tx=0).stop 2024-03-14T21:14:08.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7f81ef7fe640 1 -- 172.21.15.49:0/3394650878 reap_dead start 2024-03-14T21:14:08.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7ff467fff640 1 -- 172.21.15.49:0/1347882502 shutdown_connections 2024-03-14T21:14:08.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.272+0000 7ff467fff640 1 -- 172.21.15.49:0/1347882502 >> 172.21.15.49:0/1347882502 conn(0x7ff48406a370 msgr2=0x7ff48409fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.273+0000 7ff467fff640 1 -- 172.21.15.49:0/1347882502 shutdown_connections 2024-03-14T21:14:08.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.273+0000 7ff467fff640 1 -- 172.21.15.49:0/1347882502 wait complete. 2024-03-14T21:14:08.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.273+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 shutdown_connections 2024-03-14T21:14:08.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.273+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 >> 172.21.15.49:0/3394650878 conn(0x7f81f006adf0 msgr2=0x7f81f014a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.273+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 shutdown_connections 2024-03-14T21:14:08.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.273+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 --> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f467c01fc20 con 0x7f4668072a40 2024-03-14T21:14:08.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.273+0000 7f81d5ffb640 1 -- 172.21.15.49:0/3394650878 wait complete. 2024-03-14T21:14:08.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.274+0000 7f46817fa640 1 -- 172.21.15.49:0/3423617810 <== osd.6 v2:172.21.15.97:6800/37902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f467c01fc20 con 0x7f4668072a40 2024-03-14T21:14:08.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.274+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f4668072a40 msgr2=0x7f4668074e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.274+0000 7f466effd640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f4668072a40 0x7f4668074e60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f466806a8a0 msgr2=0x7f466806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f466effd640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f466806a8a0 0x7f466806cd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46780099d0 tx=0x7f4678009190 comp rx=0 tx=0).stop 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c03fae0 msgr2=0x7f467c020a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f466effd640 1 --2- 172.21.15.49:0/3423617810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467c03fae0 0x7f467c020a20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f467000b410 tx=0x7f467000b8d0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f46896a3640 1 -- 172.21.15.49:0/3423617810 reap_dead start 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 shutdown_connections 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 >> 172.21.15.49:0/3423617810 conn(0x7f467c00a590 msgr2=0x7f467c0422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 --> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb64803bce0 con 0x7fb634072a90 2024-03-14T21:14:08.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.275+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 shutdown_connections 2024-03-14T21:14:08.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.276+0000 7fb645ffb640 1 -- 172.21.15.49:0/2406210805 <== osd.2 v2:172.21.15.49:6824/37918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb640009530 con 0x7fb634072a90 2024-03-14T21:14:08.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.276+0000 7f466effd640 1 -- 172.21.15.49:0/3423617810 wait complete. 2024-03-14T21:14:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.276+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7fb634072a90 msgr2=0x7fb634074eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.276+0000 7fb657619640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7fb634072a90 0x7fb634074eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb63406a870 msgr2=0x7fb63406cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb63406a870 0x7fb63406cd30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb64c036000 tx=0x7fb64c03c040 comp rx=0 tx=0).stop 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb648040310 msgr2=0x7fb648175fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 --2- 172.21.15.49:0/2406210805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb648040310 0x7fb648175fb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6500687c0 tx=0x7fb650073080 comp rx=0 tx=0).stop 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb655bb6640 1 -- 172.21.15.49:0/2406210805 reap_dead start 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 shutdown_connections 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 >> 172.21.15.49:0/2406210805 conn(0x7fb648043ae0 msgr2=0x7fb6480d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 shutdown_connections 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fb657619640 1 -- 172.21.15.49:0/2406210805 wait complete. 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 --> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa57c03b8d0 con 0x7fa5680729f0 2024-03-14T21:14:08.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.277+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 --> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda4009f890 con 0x7fda20072a90 2024-03-14T21:14:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.278+0000 7fa580ff9640 1 -- 172.21.15.49:0/830457315 <== osd.4 v2:172.21.15.97:6817/37904 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa570004040 con 0x7fa5680729f0 2024-03-14T21:14:08.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.278+0000 7fda2effd640 1 -- 172.21.15.49:0/2653200172 <== osd.3 v2:172.21.15.49:6808/37909 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda4009f890 con 0x7fda20072a90 2024-03-14T21:14:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7fda20072a90 msgr2=0x7fda20074eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fda3f59e640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7fda20072a90 0x7fda20074eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7fa5680729f0 msgr2=0x7fa568074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7fa5680729f0 0x7fa568074e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa56806a7d0 msgr2=0x7fa56806cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa56806a7d0 0x7fa56806cc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa57c047380 tx=0x7fa5780079b0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa57c03cd00 msgr2=0x7fa57c15f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 --2- 172.21.15.49:0/830457315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa57c03cd00 0x7fa57c15f6a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa584063b00 tx=0x7fa584068060 comp rx=0 tx=0).stop 2024-03-14T21:14:08.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa588b84640 1 -- 172.21.15.49:0/830457315 reap_dead start 2024-03-14T21:14:08.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 shutdown_connections 2024-03-14T21:14:08.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 >> 172.21.15.49:0/830457315 conn(0x7fa57c046970 msgr2=0x7fa57c040e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda2006a870 msgr2=0x7fda2006cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fda3f59e640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda2006a870 0x7fda2006cd30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda30007fd0 tx=0x7fda30009040 comp rx=0 tx=0).stop 2024-03-14T21:14:08.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.279+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 msgr2=0x7fda400a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fda3f59e640 1 --2- 172.21.15.49:0/2653200172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda400b1240 0x7fda400a1980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda3400d520 tx=0x7fda3400d9e0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fda3ed9d640 1 -- 172.21.15.49:0/2653200172 reap_dead start 2024-03-14T21:14:08.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 shutdown_connections 2024-03-14T21:14:08.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 >> 172.21.15.49:0/2653200172 conn(0x7fda400b8b10 msgr2=0x7fda400b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 shutdown_connections 2024-03-14T21:14:08.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 --> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f18002ce0 con 0x7f7f240729c0 2024-03-14T21:14:08.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fa589385640 1 -- 172.21.15.49:0/830457315 wait complete. 2024-03-14T21:14:08.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 shutdown_connections 2024-03-14T21:14:08.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.280+0000 7fda3f59e640 1 -- 172.21.15.49:0/2653200172 wait complete. 2024-03-14T21:14:08.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.281+0000 7f7f34ff9640 1 -- 172.21.15.49:0/318410401 <== osd.7 v2:172.21.15.97:6808/37903 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f18002ce0 con 0x7f7f240729c0 2024-03-14T21:14:08.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f7f240729c0 msgr2=0x7f7f24074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f7f240729c0 0x7f7f24074de0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7f2406a7a0 msgr2=0x7f7f2406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f7f2406a7a0 0x7f7f2406cc60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f280011e0 tx=0x7f7f28009040 comp rx=0 tx=0).stop 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f380b9630 msgr2=0x7f7f3813f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 --2- 172.21.15.49:0/318410401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f380b9630 0x7f7f3813f240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c870 tx=0x7f7f3000cd30 comp rx=0 tx=0).stop 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3cb61640 1 -- 172.21.15.49:0/318410401 reap_dead start 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 shutdown_connections 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.282+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 >> 172.21.15.49:0/318410401 conn(0x7f7f380b8b10 msgr2=0x7f7f38144620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.283+0000 7ffb4e985640 1 -- 172.21.15.49:0/2136379896 --> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb400056e0 con 0x7ffb2c072920 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.283+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 shutdown_connections 2024-03-14T21:14:08.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.283+0000 7f7f3d362640 1 -- 172.21.15.49:0/318410401 wait complete. 2024-03-14T21:14:08.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.284+0000 7ffb3e7fc640 1 -- 172.21.15.49:0/2136379896 <== osd.1 v2:172.21.15.49:6800/37907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb400056e0 con 0x7ffb2c072920 2024-03-14T21:14:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.285+0000 7ffb2bfff640 1 -- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7ffb2c072920 msgr2=0x7ffb2c074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.285+0000 7ffb2bfff640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7ffb2c072920 0x7ffb2c074d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.285+0000 7ffb2bfff640 1 -- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ffb2c06a870 msgr2=0x7ffb2c06cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.285+0000 7ffb2bfff640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7ffb2c06a870 0x7ffb2c06cd30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb4015fd10 tx=0x7ffb38008040 comp rx=0 tx=0).stop 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.286+0000 7ffb2bfff640 1 -- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34004120 msgr2=0x7ffb40038c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.286+0000 7ffb2bfff640 1 --2- 172.21.15.49:0/2136379896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb34004120 0x7ffb40038c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb44005f80 tx=0x7ffb440017c0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.286+0000 7ffb4e184640 1 -- 172.21.15.49:0/2136379896 reap_dead start 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.286+0000 7ffb2bfff640 1 -- 172.21.15.49:0/2136379896 shutdown_connections 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.286+0000 7ffb2bfff640 1 -- 172.21.15.49:0/2136379896 >> 172.21.15.49:0/2136379896 conn(0x7ffb4000a560 msgr2=0x7ffb40006f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.288+0000 7ffb2bfff640 1 -- 172.21.15.49:0/2136379896 shutdown_connections 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.289+0000 7ffb2bfff640 1 -- 172.21.15.49:0/2136379896 wait complete. 2024-03-14T21:14:08.320 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:08.321 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:08.321 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-14T21:14:08.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-14T21:14:08.323 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:08.323 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-14T21:14:08.324 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:08.324 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-14T21:14:08.325 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:08.325 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-14T21:14:08.326 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:08.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-14T21:14:08.327 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:08.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-14T21:14:08.328 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:08.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-14T21:14:08.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.560+0000 7f47db9bb640 1 Processor -- start 2024-03-14T21:14:08.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.560+0000 7f47db9bb640 1 -- start start 2024-03-14T21:14:08.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.561+0000 7f227694e640 1 Processor -- start 2024-03-14T21:14:08.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.561+0000 7f227694e640 1 -- start start 2024-03-14T21:14:08.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.561+0000 7f47db9bb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d4154420 0x7f47d4156810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.561+0000 7f47db9bb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d414b090 con 0x7f47d414b520 2024-03-14T21:14:08.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.561+0000 7f47db9bb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d40b82b0 con 0x7f47d4145060 2024-03-14T21:14:08.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.561+0000 7f47db9bb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d406d9d0 con 0x7f47d4154420 2024-03-14T21:14:08.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f47d8f56640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f47d414b520 0x7f47d40b7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36892/0 (socket says 172.21.15.49:36892) 2024-03-14T21:14:08.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f47d8f56640 1 -- 172.21.15.49:0/918213175 learned_addr learned my addr 172.21.15.49:0/918213175 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:08.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f96aa804640 1 Processor -- start 2024-03-14T21:14:08.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f96aa804640 1 -- start start 2024-03-14T21:14:08.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f227694e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22700b7f50 0x7f2270147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f47d9f58640 1 --2- 172.21.15.49:0/918213175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d4154420 0x7f47d4156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f227694e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22701480f0 0x7f2270151040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f226f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22700b7f50 0x7f2270147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f227694e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700b8390 con 0x7f22701409a0 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f227694e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22701466e0 con 0x7f22700b7f50 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f226f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22700b7f50 0x7f2270147bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37524/0 (socket says 172.21.15.49:37524) 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f226f7fe640 1 -- 172.21.15.49:0/4027417403 learned_addr learned my addr 172.21.15.49:0/4027417403 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f227694e640 1 -- 172.21.15.49:0/4027417403 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22700b1ec0 con 0x7f22701480f0 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.562+0000 7f226ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f22701409a0 0x7f22700b17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47772/0 (socket says 172.21.15.49:47772) 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f2274eeb640 1 --2- 172.21.15.49:0/4027417403 >> v2:172.21.15.97:3300/0 conn(0x7f22701480f0 0x7f2270151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f2274eeb640 1 -- 172.21.15.49:0/4027417403 >> v2:172.21.15.49:3300/0 conn(0x7f22700b7f50 msgr2=0x7f2270147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f2274eeb640 1 --2- 172.21.15.49:0/4027417403 >> v2:172.21.15.49:3300/0 conn(0x7f22700b7f50 0x7f2270147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f47cbfff640 1 -- 172.21.15.49:0/918213175 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709031176 0 0) 0x7f47d414b090 con 0x7f47d414b520 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f96aa804640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a4144b70 0x7f96a406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f96aa804640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f96a409ece0 0x7f96a40b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f96aa804640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a4144f50 con 0x7f96a414a810 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f96aa804640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a406b890 con 0x7f96a409ece0 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f96aa804640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a40b89f0 con 0x7f96a4144b70 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f226effd640 1 -- 172.21.15.49:0/4027417403 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578660076 0 0) 0x7f22700b8390 con 0x7f22701409a0 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f2274eeb640 1 -- 172.21.15.49:0/4027417403 >> v1:172.21.15.97:6789/0 conn(0x7f22701409a0 legacy=0x7f22700b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f2274eeb640 1 -- 172.21.15.49:0/4027417403 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2270147690 con 0x7f22701480f0 2024-03-14T21:14:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f47cbfff640 1 -- 172.21.15.49:0/918213175 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c0003520 con 0x7f47d414b520 2024-03-14T21:14:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f47d9f58640 1 -- 172.21.15.49:0/918213175 >> v1:172.21.15.97:6789/0 conn(0x7f47d4145060 legacy=0x7f47d406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f47d9f58640 1 -- 172.21.15.49:0/918213175 >> v1:172.21.15.49:6789/0 conn(0x7f47d414b520 legacy=0x7f47d40b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f47d9f58640 1 -- 172.21.15.49:0/918213175 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d406c040 con 0x7f47d4154420 2024-03-14T21:14:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.563+0000 7f96a8da1640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f96a414a810 0x7f96a414dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47786/0 (socket says 172.21.15.49:47786) 2024-03-14T21:14:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f96a409ece0 0x7f96a40b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f96a409ece0 0x7f96a40b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37538/0 (socket says 172.21.15.49:37538) 2024-03-14T21:14:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a8da1640 1 -- 172.21.15.49:0/527459831 learned_addr learned my addr 172.21.15.49:0/527459831 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:08.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f6231a6c640 1 Processor -- start 2024-03-14T21:14:08.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a3fff640 1 --2- 172.21.15.49:0/527459831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a4144b70 0x7f96a406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a37fe640 1 -- 172.21.15.49:0/527459831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a4144b70 msgr2=0x7f96a406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a37fe640 1 --2- 172.21.15.49:0/527459831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a4144b70 0x7f96a406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a37fe640 1 -- 172.21.15.49:0/527459831 >> v1:172.21.15.97:6789/0 conn(0x7f96a414a810 legacy=0x7f96a414dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f96a37fe640 1 -- 172.21.15.49:0/527459831 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a40bcf90 con 0x7f96a409ece0 2024-03-14T21:14:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.564+0000 7f6231a6c640 1 -- start start 2024-03-14T21:14:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f2274eeb640 1 --2- 172.21.15.49:0/4027417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f2270151040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22700b25c0 tx=0x7f226802bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55225a51305ab96 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f47d9f58640 1 --2- 172.21.15.49:0/918213175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d4154420 0x7f47d4156810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009dc0 tx=0x7f47cc02bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf076bdd4495320 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f226effd640 1 -- 172.21.15.49:0/4027417403 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226802cc70 con 0x7f22701480f0 2024-03-14T21:14:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f226effd640 1 -- 172.21.15.49:0/4027417403 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226802cdd0 con 0x7f22701480f0 2024-03-14T21:14:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f226effd640 1 -- 172.21.15.49:0/4027417403 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268034660 con 0x7f22701480f0 2024-03-14T21:14:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f47cbfff640 1 -- 172.21.15.49:0/918213175 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc02ccc0 con 0x7f47d4154420 2024-03-14T21:14:08.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f47cbfff640 1 -- 172.21.15.49:0/918213175 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc02ce20 con 0x7f47d4154420 2024-03-14T21:14:08.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f6231a6c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6224006ec0 0x7f622403c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f6231a6c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62240d53e0 0x7f622403e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f6231a6c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622403ebd0 con 0x7f62240072a0 2024-03-14T21:14:08.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f6231a6c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622403a5d0 con 0x7f62240d53e0 2024-03-14T21:14:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.565+0000 7f6231a6c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622403a8b0 con 0x7f6224006ec0 2024-03-14T21:14:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f622bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62240d53e0 0x7f622403e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f622bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62240d53e0 0x7f622403e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37550/0 (socket says 172.21.15.49:37550) 2024-03-14T21:14:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f622bfff640 1 -- 172.21.15.49:0/4025900300 learned_addr learned my addr 172.21.15.49:0/4025900300 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f47cbfff640 1 -- 172.21.15.49:0/918213175 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc034660 con 0x7f47d4154420 2024-03-14T21:14:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f227694e640 1 -- 172.21.15.49:0/4027417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 msgr2=0x7f2270151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f227694e640 1 --2- 172.21.15.49:0/4027417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f2270151040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22700b25c0 tx=0x7f226802bec0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f47db9bb640 1 -- 172.21.15.49:0/918213175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d4154420 msgr2=0x7f47d4156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f47db9bb640 1 --2- 172.21.15.49:0/918213175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d4154420 0x7f47d4156810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009dc0 tx=0x7f47cc02bf10 comp rx=0 tx=0).stop 2024-03-14T21:14:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f6230a6a640 1 --2- 172.21.15.49:0/4025900300 >> v2:172.21.15.97:3300/0 conn(0x7f6224006ec0 0x7f622403c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f622bfff640 1 -- 172.21.15.49:0/4025900300 >> v2:172.21.15.97:3300/0 conn(0x7f6224006ec0 msgr2=0x7f622403c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f622bfff640 1 --2- 172.21.15.49:0/4025900300 >> v2:172.21.15.97:3300/0 conn(0x7f6224006ec0 0x7f622403c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f622bfff640 1 -- 172.21.15.49:0/4025900300 >> v1:172.21.15.49:6789/0 conn(0x7f62240072a0 legacy=0x7f62240d8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f96a37fe640 1 --2- 172.21.15.49:0/527459831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a409ece0 0x7f96a40b7a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96900097a0 tx=0x7f969002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b3b6c1d2792580 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f527901f640 1 Processor -- start 2024-03-14T21:14:08.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f574552c640 1 Processor -- start 2024-03-14T21:14:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f527901f640 1 -- start start 2024-03-14T21:14:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f96a2ffd640 1 -- 172.21.15.49:0/527459831 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690031830 con 0x7f96a409ece0 2024-03-14T21:14:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f96a2ffd640 1 -- 172.21.15.49:0/527459831 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9690031990 con 0x7f96a409ece0 2024-03-14T21:14:08.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f96a2ffd640 1 -- 172.21.15.49:0/527459831 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690031d20 con 0x7f96a409ece0 2024-03-14T21:14:08.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f622bfff640 1 -- 172.21.15.49:0/4025900300 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622403ab90 con 0x7f62240d53e0 2024-03-14T21:14:08.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f6230a6a640 1 --2- 172.21.15.49:0/4025900300 >> v2:172.21.15.97:3300/0 conn(0x7f6224006ec0 0x7f622403c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:08.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f581eff9640 1 Processor -- start 2024-03-14T21:14:08.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f574552c640 1 -- start start 2024-03-14T21:14:08.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f581eff9640 1 -- start start 2024-03-14T21:14:08.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f91aaafa640 1 Processor -- start 2024-03-14T21:14:08.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f527901f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 0x7f526c0d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f527901f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526c02c3b0 con 0x7f526c0074a0 2024-03-14T21:14:08.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f527901f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526c03b5f0 con 0x7f526c0d1c40 2024-03-14T21:14:08.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f527901f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c03be90 con 0x7f526c042a40 2024-03-14T21:14:08.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f96aa804640 1 -- 172.21.15.49:0/527459831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a409ece0 msgr2=0x7f96a40b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f96aa804640 1 --2- 172.21.15.49:0/527459831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a409ece0 0x7f96a40b7a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96900097a0 tx=0x7f969002b8d0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f574552c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f57380d53e0 0x7f573803c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f574552c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573803a5d0 con 0x7f57380072a0 2024-03-14T21:14:08.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f574552c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573803a8b0 con 0x7f5738006ec0 2024-03-14T21:14:08.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5273fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f526c0074a0 0x7f526c03cc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36916/0 (socket says 172.21.15.49:36916) 2024-03-14T21:14:08.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5273fff640 1 -- 172.21.15.49:0/475667598 learned_addr learned my addr 172.21.15.49:0/475667598 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:08.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f574552c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573803ab90 con 0x7f57380d53e0 2024-03-14T21:14:08.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f96aa804640 1 -- 172.21.15.49:0/527459831 shutdown_connections 2024-03-14T21:14:08.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f96aa804640 1 --2- 172.21.15.49:0/527459831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a409ece0 0x7f96a40b7a90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f96aa804640 1 --2- 172.21.15.49:0/527459831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a4144b70 0x7f96a406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f96aa804640 1 -- 172.21.15.49:0/527459831 >> 172.21.15.49:0/527459831 conn(0x7f96a40b8b80 msgr2=0x7f96a40b8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5744d2b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f57380072a0 0x7f57380dfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36922/0 (socket says 172.21.15.49:36922) 2024-03-14T21:14:08.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5744d2b640 1 -- 172.21.15.49:0/2033159900 learned_addr learned my addr 172.21.15.49:0/2033159900 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:08.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f52737fe640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 0x7f526c0d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f573f7fe640 1 --2- 172.21.15.49:0/2033159900 >> v2:172.21.15.97:3300/0 conn(0x7f57380d53e0 0x7f573803c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f47db9bb640 1 -- 172.21.15.49:0/918213175 shutdown_connections 2024-03-14T21:14:08.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f47db9bb640 1 --2- 172.21.15.49:0/918213175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d4154420 0x7f47d4156810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.566+0000 7f47db9bb640 1 -- 172.21.15.49:0/918213175 >> 172.21.15.49:0/918213175 conn(0x7f47d4069ce0 msgr2=0x7f47d406a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f47db9bb640 1 -- 172.21.15.49:0/918213175 shutdown_connections 2024-03-14T21:14:08.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f47db9bb640 1 -- 172.21.15.49:0/918213175 wait complete. 2024-03-14T21:14:08.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f227694e640 1 -- 172.21.15.49:0/4027417403 shutdown_connections 2024-03-14T21:14:08.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f227694e640 1 --2- 172.21.15.49:0/4027417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f2270151040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f227694e640 1 --2- 172.21.15.49:0/4027417403 >> v2:172.21.15.49:3300/0 conn(0x7f22700b7f50 0x7f2270147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f227694e640 1 -- 172.21.15.49:0/4027417403 >> 172.21.15.49:0/4027417403 conn(0x7f22700b8b80 msgr2=0x7f22700b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f47db9bb640 1 Processor -- start 2024-03-14T21:14:08.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090041491 0 0) 0x7f526c03b5f0 con 0x7f526c0d1c40 2024-03-14T21:14:08.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f581eff9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58180bc510 0x7f581814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f525c003520 con 0x7f526c0d1c40 2024-03-14T21:14:08.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801647757 0 0) 0x7f526c02c3b0 con 0x7f526c0074a0 2024-03-14T21:14:08.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f526c03b5f0 con 0x7f526c0074a0 2024-03-14T21:14:08.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f227694e640 1 -- 172.21.15.49:0/4027417403 shutdown_connections 2024-03-14T21:14:08.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f573effd640 1 -- 172.21.15.49:0/2033159900 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935790952 0 0) 0x7f573803a5d0 con 0x7f57380072a0 2024-03-14T21:14:08.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f573effd640 1 -- 172.21.15.49:0/2033159900 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5730003520 con 0x7f57380072a0 2024-03-14T21:14:08.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f227694e640 1 -- 172.21.15.49:0/4027417403 wait complete. 2024-03-14T21:14:08.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.567+0000 7f91aaafa640 1 -- start start 2024-03-14T21:14:08.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f91aaafa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91a40bac30 0x7f91a414f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f91aaafa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40a4c70 0x7f91a4159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f581eff9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 0x7f581814e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f581dff7640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5818144c00 0x7f58180bcb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36934/0 (socket says 172.21.15.49:36934) 2024-03-14T21:14:08.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91aaafa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a40a40d0 con 0x7f91a414fa40 2024-03-14T21:14:08.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91aaafa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a40a3b50 con 0x7f91a40bac30 2024-03-14T21:14:08.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91aaafa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a40a3730 con 0x7f91a40a4c70 2024-03-14T21:14:08.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581dff7640 1 -- 172.21.15.49:0/2452376186 learned_addr learned my addr 172.21.15.49:0/2452376186 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:08.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3571646327 0 0) 0x7f525c003520 con 0x7f526c0d1c40 2024-03-14T21:14:08.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 msgr2=0x7f526c0d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 0x7f526c0d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 >> v1:172.21.15.49:6789/0 conn(0x7f526c0074a0 legacy=0x7f526c03cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58180be970 con 0x7f5818144c00 2024-03-14T21:14:08.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f96aa804640 1 -- 172.21.15.49:0/527459831 shutdown_connections 2024-03-14T21:14:08.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f573f7fe640 1 -- 172.21.15.49:0/2033159900 >> v1:172.21.15.97:6789/0 conn(0x7f5738006ec0 legacy=0x7f573803e690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T21:14:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f573f7fe640 1 -- 172.21.15.49:0/2033159900 >> v1:172.21.15.49:6789/0 conn(0x7f57380072a0 legacy=0x7f57380dfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f573f7fe640 1 -- 172.21.15.49:0/2033159900 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573803e170 con 0x7f57380d53e0 2024-03-14T21:14:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91a9af8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91a40bac30 0x7f91a414f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91a9af8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91a40bac30 0x7f91a414f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33838/0 (socket says 172.21.15.49:33838) 2024-03-14T21:14:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91a9af8640 1 -- 172.21.15.49:0/3280204404 learned_addr learned my addr 172.21.15.49:0/3280204404 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91aa2f9640 1 --2- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40a4c70 0x7f91a4159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c03b8d0 con 0x7f526c0d1c40 2024-03-14T21:14:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f527404e550 con 0x7f526c0d1c40 2024-03-14T21:14:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5818144fe0 con 0x7f58180bc510 2024-03-14T21:14:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 >> v1:172.21.15.97:6789/0 conn(0x7f526c0d1c40 legacy=0x7f526c0db8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c004b60 0x7f525c004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58180be3f0 con 0x7f58180bae90 2024-03-14T21:14:08.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581d7f6640 1 --2- 172.21.15.49:0/2452376186 >> v2:172.21.15.97:3300/0 conn(0x7f58180bc510 0x7f581814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.568+0000 7f622bfff640 1 --2- 172.21.15.49:0/4025900300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f622403e690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f622403fd50 tx=0x7f621802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861670cc3e37540d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f525c003760 0x7f525c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 0x7f525c004120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c03be90 con 0x7f525c003760 2024-03-14T21:14:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c00ba00 con 0x7f525c004b60 2024-03-14T21:14:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c00bb40 con 0x7f526c042a40 2024-03-14T21:14:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f5273fff640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c004b60 0x7f525c004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f527881e640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 0x7f525c004120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f52737fe640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f525c003760 0x7f525c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f47db9bb640 1 -- start start 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65982114 0 0) 0x7f58180be970 con 0x7f5818144c00 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5804003520 con 0x7f5818144c00 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f581e7f8640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 0x7f581814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2924252511 0 0) 0x7f5804003520 con 0x7f5818144c00 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 msgr2=0x7f581814e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 0x7f581814e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 >> v2:172.21.15.97:3300/0 conn(0x7f58180bc510 msgr2=0x7f581814dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 --2- 172.21.15.49:0/2452376186 >> v2:172.21.15.97:3300/0 conn(0x7f58180bc510 0x7f581814dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180bd230 con 0x7f5818144c00 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f5808001670 con 0x7f5818144c00 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 >> v1:172.21.15.49:6789/0 conn(0x7f5818144c00 legacy=0x7f58180bcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804003970 0x7f5804003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58040047c0 0x7f5804004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 0x7f5804005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180be3f0 con 0x7f5804003970 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580400b980 con 0x7f58040047c0 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580400bac0 con 0x7f58180bae90 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581d7f6640 1 --2- 172.21.15.49:0/2452376186 >> v2:172.21.15.97:3300/0 conn(0x7f58180bc510 0x7f581814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:08.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f96aa804640 1 -- 172.21.15.49:0/527459831 wait complete. 2024-03-14T21:14:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91a9af8640 1 -- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40a4c70 msgr2=0x7f91a4159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91a9af8640 1 --2- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40a4c70 0x7f91a4159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91a9af8640 1 -- 172.21.15.49:0/3280204404 >> v1:172.21.15.97:6789/0 conn(0x7f91a414fa40 legacy=0x7f91a40a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.569+0000 7f91a9af8640 1 -- 172.21.15.49:0/3280204404 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a414fe60 con 0x7f91a40bac30 2024-03-14T21:14:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581e7f8640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 0x7f5804005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f5273fff640 1 -- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 msgr2=0x7f525c004120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f5273fff640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f526c042a40 0x7f525c004120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f5273fff640 1 -- 172.21.15.49:0/475667598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f525c003760 msgr2=0x7f525c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f5273fff640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f525c003760 0x7f525c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f5273fff640 1 -- 172.21.15.49:0/475667598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c03b8d0 con 0x7f525c004b60 2024-03-14T21:14:08.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f227694e640 1 Processor -- start 2024-03-14T21:14:08.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581d7f6640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58040047c0 0x7f5804004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f581dff7640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804003970 0x7f5804003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f47db9bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d4145060 0x7f47d4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f47db9bb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d414b520 0x7f47d4149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f47db9bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d4154420 0x7f47d41e3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f47db9bb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d414a1f0 con 0x7f47d4145060 2024-03-14T21:14:08.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f47db9bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b6060 con 0x7f47d4154420 2024-03-14T21:14:08.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f47db9bb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40bbc00 con 0x7f47d414b520 2024-03-14T21:14:08.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f96aa804640 1 Processor -- start 2024-03-14T21:14:08.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f581d7f6640 1 -- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 msgr2=0x7f5804005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f91a9af8640 1 --2- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a40bac30 0x7f91a414f500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009870 tx=0x7f91a002b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ac15312096e863 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f581d7f6640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58180bae90 0x7f5804005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f581d7f6640 1 -- 172.21.15.49:0/2452376186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804003970 msgr2=0x7f5804003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:08.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f581d7f6640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5804003970 0x7f5804003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f581d7f6640 1 -- 172.21.15.49:0/2452376186 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180bd230 con 0x7f58040047c0 2024-03-14T21:14:08.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f47d8f56640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d414b520 0x7f47d4149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f47d8f56640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d414b520 0x7f47d4149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46526/0 (socket says 172.21.15.49:46526) 2024-03-14T21:14:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f47d8f56640 1 -- 172.21.15.49:0/775509863 learned_addr learned my addr 172.21.15.49:0/775509863 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f622b7fe640 1 -- 172.21.15.49:0/4025900300 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218030860 con 0x7f62240d53e0 2024-03-14T21:14:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f622b7fe640 1 -- 172.21.15.49:0/4025900300 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62180309c0 con 0x7f62240d53e0 2024-03-14T21:14:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f622b7fe640 1 -- 172.21.15.49:0/4025900300 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218030d50 con 0x7f62240d53e0 2024-03-14T21:14:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f91a8af6640 1 -- 172.21.15.49:0/3280204404 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0031740 con 0x7f91a40bac30 2024-03-14T21:14:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f91a8af6640 1 -- 172.21.15.49:0/3280204404 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a00318a0 con 0x7f91a40bac30 2024-03-14T21:14:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f91a8af6640 1 -- 172.21.15.49:0/3280204404 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0031c30 con 0x7f91a40bac30 2024-03-14T21:14:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f47d9f58640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d4154420 0x7f47d41e3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f227694e640 1 -- start start 2024-03-14T21:14:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f47d9757640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d4145060 0x7f47d4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f5273fff640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c004b60 0x7f525c004f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5268001940 tx=0x7f525c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ecbfbf788e78167 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f47d9f58640 1 -- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d414b520 msgr2=0x7f47d4149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f47d9f58640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d414b520 0x7f47d4149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47d9f58640 1 -- 172.21.15.49:0/775509863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d4145060 msgr2=0x7f47d4070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47d9f58640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d4145060 0x7f47d4070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47d9f58640 1 -- 172.21.15.49:0/775509863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009a70 con 0x7f47d4154420 2024-03-14T21:14:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47d8f56640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d414b520 0x7f47d4149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f6231a6c640 1 -- 172.21.15.49:0/4025900300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 msgr2=0x7f622403e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f6231a6c640 1 --2- 172.21.15.49:0/4025900300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f622403e690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f622403fd50 tx=0x7f621802b9a0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.570+0000 7f573f7fe640 1 --2- 172.21.15.49:0/2033159900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 0x7f573803c430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f57380287d0 tx=0x7f572c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c40335004d123c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f573effd640 1 -- 172.21.15.49:0/2033159900 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c02d040 con 0x7f57380d53e0 2024-03-14T21:14:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f573effd640 1 -- 172.21.15.49:0/2033159900 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c031810 con 0x7f57380d53e0 2024-03-14T21:14:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268004aa0 con 0x7f525c004b60 2024-03-14T21:14:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526802d040 con 0x7f525c004b60 2024-03-14T21:14:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f5272ffd640 1 -- 172.21.15.49:0/475667598 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268004e40 con 0x7f525c004b60 2024-03-14T21:14:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f227694e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22700b7f50 0x7f22700a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47d9f58640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d4154420 0x7f47d41e3930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47d406af90 tx=0x7f47cc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f227694e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22701409a0 0x7f22700b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f573effd640 1 -- 172.21.15.49:0/2033159900 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c031ba0 con 0x7f57380d53e0 2024-03-14T21:14:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f227694e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f227013e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.571+0000 7f96aa804640 1 -- start start 2024-03-14T21:14:08.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96aa804640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a409ece0 0x7f96a4071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f574552c640 1 -- 172.21.15.49:0/2033159900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 msgr2=0x7f573803c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f574552c640 1 --2- 172.21.15.49:0/2033159900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 0x7f573803c430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f57380287d0 tx=0x7f572c02b8b0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f227694e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227013fd80 con 0x7f22701409a0 2024-03-14T21:14:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47c9ffb640 1 -- 172.21.15.49:0/775509863 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc03e610 con 0x7f47d4154420 2024-03-14T21:14:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96aa804640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a4144b70 0x7f96a40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f574552c640 1 -- 172.21.15.49:0/2033159900 shutdown_connections 2024-03-14T21:14:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f574552c640 1 --2- 172.21.15.49:0/2033159900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 0x7f573803c430 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f574552c640 1 -- 172.21.15.49:0/2033159900 >> 172.21.15.49:0/2033159900 conn(0x7f573803edf0 msgr2=0x7f573803f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f227694e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700a0a30 con 0x7f22701480f0 2024-03-14T21:14:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f227694e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227006bc90 con 0x7f22700b7f50 2024-03-14T21:14:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47c9ffb640 1 -- 172.21.15.49:0/775509863 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc03e770 con 0x7f47d4154420 2024-03-14T21:14:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f6231a6c640 1 -- 172.21.15.49:0/4025900300 shutdown_connections 2024-03-14T21:14:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f6231a6c640 1 --2- 172.21.15.49:0/4025900300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f622403e690 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f6231a6c640 1 --2- 172.21.15.49:0/4025900300 >> v2:172.21.15.97:3300/0 conn(0x7f6224006ec0 0x7f622403c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f6231a6c640 1 -- 172.21.15.49:0/4025900300 >> 172.21.15.49:0/4025900300 conn(0x7f622403edf0 msgr2=0x7f622403f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f2274eeb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f227013e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f2274eeb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f227013e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33874/0 (socket says 172.21.15.49:33874) 2024-03-14T21:14:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f2274eeb640 1 -- 172.21.15.49:0/2353728913 learned_addr learned my addr 172.21.15.49:0/2353728913 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47c9ffb640 1 -- 172.21.15.49:0/775509863 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc03eb00 con 0x7f47d4154420 2024-03-14T21:14:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96aa804640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a414a810 0x7f96a4098bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96aa804640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a4071650 con 0x7f96a4144b70 2024-03-14T21:14:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96aa804640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a4099820 con 0x7f96a409ece0 2024-03-14T21:14:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96aa804640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a406f260 con 0x7f96a414a810 2024-03-14T21:14:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f6231a6c640 1 -- 172.21.15.49:0/4025900300 shutdown_connections 2024-03-14T21:14:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f226ffff640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22700b7f50 0x7f22700a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f47db9bb640 1 -- 172.21.15.49:0/775509863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d40b5dc0 con 0x7f47d4154420 2024-03-14T21:14:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a409ece0 0x7f96a4071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a409ece0 0x7f96a4071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33876/0 (socket says 172.21.15.49:33876) 2024-03-14T21:14:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f96a3fff640 1 -- 172.21.15.49:0/2177679077 learned_addr learned my addr 172.21.15.49:0/2177679077 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a8da1640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a414a810 0x7f96a4098bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f527901f640 1 -- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c004b60 msgr2=0x7f525c004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f527901f640 1 --2- 172.21.15.49:0/475667598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c004b60 0x7f525c004f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5268001940 tx=0x7f525c0057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f527881e640 1 -- 172.21.15.49:0/475667598 reap_dead start 2024-03-14T21:14:08.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f226ffff640 1 -- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 msgr2=0x7f227013e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f226ffff640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f227013e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f226ffff640 1 -- 172.21.15.49:0/2353728913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22701409a0 msgr2=0x7f22700b9e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T21:14:08.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f527901f640 1 -- 172.21.15.49:0/475667598 shutdown_connections 2024-03-14T21:14:08.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f527901f640 1 -- 172.21.15.49:0/475667598 >> 172.21.15.49:0/475667598 conn(0x7f526c043220 msgr2=0x7f526c043600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f226ffff640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22701409a0 0x7f22700b9e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f91aaafa640 1 -- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a40bac30 msgr2=0x7f91a414f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f91aaafa640 1 --2- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a40bac30 0x7f91a414f500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009870 tx=0x7f91a002b7e0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f226ffff640 1 -- 172.21.15.49:0/2353728913 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268009a70 con 0x7f22700b7f50 2024-03-14T21:14:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f581d7f6640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58040047c0 0x7f5804004c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5814009c90 tx=0x7f58040057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d374384d186b16a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581402c980 con 0x7f58040047c0 2024-03-14T21:14:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.572+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581402cae0 con 0x7f58040047c0 2024-03-14T21:14:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f527901f640 1 -- 172.21.15.49:0/475667598 shutdown_connections 2024-03-14T21:14:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f6231a6c640 1 -- 172.21.15.49:0/4025900300 wait complete. 2024-03-14T21:14:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f91aaafa640 1 -- 172.21.15.49:0/3280204404 shutdown_connections 2024-03-14T21:14:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f91aaafa640 1 --2- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40a4c70 0x7f91a4159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f91aaafa640 1 --2- 172.21.15.49:0/3280204404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a40bac30 0x7f91a414f500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f91aaafa640 1 -- 172.21.15.49:0/3280204404 >> 172.21.15.49:0/3280204404 conn(0x7f91a40bcac0 msgr2=0x7f91a40bced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f47db9bb640 1 -- 172.21.15.49:0/775509863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d40bb890 con 0x7f47d4154420 2024-03-14T21:14:08.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a37fe640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a4144b70 0x7f96a40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f226ffff640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22700b7f50 0x7f22700a04f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f226400c450 tx=0x7f226400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a8da1640 1 -- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a409ece0 msgr2=0x7f96a4071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a8da1640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a409ece0 0x7f96a4071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a8da1640 1 -- 172.21.15.49:0/2177679077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a4144b70 msgr2=0x7f96a40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a8da1640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a4144b70 0x7f96a40703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a8da1640 1 -- 172.21.15.49:0/2177679077 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009400 con 0x7f96a414a810 2024-03-14T21:14:08.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f91aaafa640 1 -- 172.21.15.49:0/3280204404 shutdown_connections 2024-03-14T21:14:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f226cff9640 1 -- 172.21.15.49:0/2353728913 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640074e0 con 0x7f22700b7f50 2024-03-14T21:14:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f96a37fe640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a4144b70 0x7f96a40703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f574552c640 1 -- 172.21.15.49:0/2033159900 shutdown_connections 2024-03-14T21:14:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f527901f640 1 -- 172.21.15.49:0/475667598 wait complete. 2024-03-14T21:14:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f226cff9640 1 -- 172.21.15.49:0/2353728913 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2264007640 con 0x7f22700b7f50 2024-03-14T21:14:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58040047c0 msgr2=0x7f5804004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f581cff5640 1 -- 172.21.15.49:0/2452376186 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581402c980 con 0x7f58040047c0 2024-03-14T21:14:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.573+0000 7f581eff9640 1 --2- 172.21.15.49:0/2452376186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58040047c0 0x7f5804004c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5814009c90 tx=0x7f58040057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f581e7f8640 1 -- 172.21.15.49:0/2452376186 reap_dead start 2024-03-14T21:14:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 shutdown_connections 2024-03-14T21:14:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 >> 172.21.15.49:0/2452376186 conn(0x7f58180a1e20 msgr2=0x7f58180a2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f91aaafa640 1 -- 172.21.15.49:0/3280204404 wait complete. 2024-03-14T21:14:08.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f96a8da1640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a414a810 0x7f96a4098bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f969c00cb20 tx=0x7f969c00cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f574552c640 1 -- 172.21.15.49:0/2033159900 wait complete. 2024-03-14T21:14:08.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f527901f640 1 Processor -- start 2024-03-14T21:14:08.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f226cff9640 1 -- 172.21.15.49:0/2353728913 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264004a20 con 0x7f22700b7f50 2024-03-14T21:14:08.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 shutdown_connections 2024-03-14T21:14:08.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f581eff9640 1 -- 172.21.15.49:0/2452376186 wait complete. 2024-03-14T21:14:08.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f527901f640 1 -- start start 2024-03-14T21:14:08.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22700bb2a0 con 0x7f22700b7f50 2024-03-14T21:14:08.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22700bb730 con 0x7f22700b7f50 2024-03-14T21:14:08.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f47db9bb640 1 -- 172.21.15.49:0/775509863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b4005250 con 0x7f47d4154420 2024-03-14T21:14:08.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f47c9ffb640 1 -- 172.21.15.49:0/775509863 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f47cc03b050 con 0x7f47d4154420 2024-03-14T21:14:08.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f91aaafa640 1 Processor -- start 2024-03-14T21:14:08.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f6231a6c640 1 Processor -- start 2024-03-14T21:14:08.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f527901f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f525c004b60 0x7f526c02a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.574+0000 7f6231a6c640 1 -- start start 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f6231a6c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6224006ec0 0x7f62240d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f6231a6c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62240072a0 0x7f62240cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f6231a6c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f62240c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f6231a6c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62240d90b0 con 0x7f62240d53e0 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f6231a6c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62240cbf30 con 0x7f62240072a0 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f6231a6c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62240caa10 con 0x7f6224006ec0 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62240072a0 0x7f62240cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f581eff9640 1 Processor -- start 2024-03-14T21:14:08.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f581eff9640 1 -- start start 2024-03-14T21:14:08.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62240072a0 0x7f62240cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33886/0 (socket says 172.21.15.49:33886) 2024-03-14T21:14:08.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 -- 172.21.15.49:0/2757711833 learned_addr learned my addr 172.21.15.49:0/2757711833 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f623126b640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f62240c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 -- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6224006ec0 msgr2=0x7f62240d8b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T21:14:08.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6224006ec0 0x7f62240d8b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 -- 172.21.15.49:0/2757711833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 msgr2=0x7f62240c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f527901f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526c02b050 0x7f526c163bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f5273fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f525c004b60 0x7f526c02a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f96a0ff9640 1 -- 172.21.15.49:0/2177679077 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00abd0 con 0x7f96a414a810 2024-03-14T21:14:08.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f96a0ff9640 1 -- 172.21.15.49:0/2177679077 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c00ad30 con 0x7f96a414a810 2024-03-14T21:14:08.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.575+0000 7f96a0ff9640 1 -- 172.21.15.49:0/2177679077 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00aea0 con 0x7f96a414a810 2024-03-14T21:14:08.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f5273fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f525c004b60 0x7f526c02a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46552/0 (socket says 172.21.15.49:46552) 2024-03-14T21:14:08.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f527901f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f526c164110 0x7f526c028b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f47c9ffb640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f47b806a870 0x7f47b806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f62240c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f622bfff640 1 -- 172.21.15.49:0/2757711833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180094d0 con 0x7f62240072a0 2024-03-14T21:14:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f623126b640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f62240c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f527901f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c02a8f0 con 0x7f526c164110 2024-03-14T21:14:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f527901f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c0cec00 con 0x7f526c02b050 2024-03-14T21:14:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f527901f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c16a160 con 0x7f525c004b60 2024-03-14T21:14:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f5273fff640 1 -- 172.21.15.49:0/238146513 learned_addr learned my addr 172.21.15.49:0/238146513 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f22567fc640 1 -- 172.21.15.49:0/2353728913 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22700670d0 con 0x7f22700b7f50 2024-03-14T21:14:08.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f527881e640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f526c164110 0x7f526c028b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f52737fe640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526c02b050 0x7f526c163bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f622bfff640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62240072a0 0x7f62240cfda0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6218000c00 tx=0x7f6218004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f47d9757640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f47b806a870 0x7f47b806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f47c9ffb640 1 -- 172.21.15.49:0/775509863 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47cc0ab570 con 0x7f47d4154420 2024-03-14T21:14:08.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f574552c640 1 Processor -- start 2024-03-14T21:14:08.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.576+0000 7f574552c640 1 -- start start 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f574552c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5738006ec0 0x7f573801d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f574552c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57380072a0 0x7f57380d1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f574552c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 0x7f57380d1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f574552c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57381613d0 con 0x7f5738006ec0 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f574552c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57380d2db0 con 0x7f57380d53e0 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f574552c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573801d920 con 0x7f57380072a0 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f573ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5738006ec0 0x7f573801d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f573ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5738006ec0 0x7f573801d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37604/0 (socket says 172.21.15.49:37604) 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f573ffff640 1 -- 172.21.15.49:0/3774134480 learned_addr learned my addr 172.21.15.49:0/3774134480 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f581eff9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58040047c0 0x7f58181453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f5273fff640 1 -- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526c02b050 msgr2=0x7f526c163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f5273fff640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526c02b050 0x7f526c163bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f581eff9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581814e580 0x7f58180b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f5273fff640 1 -- 172.21.15.49:0/238146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f526c164110 msgr2=0x7f526c028b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f5273fff640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f526c164110 0x7f526c028b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f5273fff640 1 -- 172.21.15.49:0/238146513 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c16a400 con 0x7f525c004b60 2024-03-14T21:14:08.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f47d9757640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f47b806a870 0x7f47b806cd30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47d00016c0 tx=0x7f47d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f5744d2b640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 0x7f57380d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581dff7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58040047c0 0x7f58181453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581dff7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58040047c0 0x7f58181453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46570/0 (socket says 172.21.15.49:46570) 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581eff9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818145a80 0x7f58181e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581eff9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58181458e0 con 0x7f581814e580 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581eff9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180ba300 con 0x7f5818145a80 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581eff9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5818147cf0 con 0x7f58040047c0 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f226cff9640 1 -- 172.21.15.49:0/2353728913 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2264004b80 con 0x7f22700b7f50 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f226cff9640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f225c06a7a0 0x7f225c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f573ffff640 1 -- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57380072a0 msgr2=0x7f57380d1240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f573ffff640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57380072a0 0x7f57380d1240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f573ffff640 1 -- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 msgr2=0x7f57380d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f573ffff640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 0x7f57380d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f573ffff640 1 -- 172.21.15.49:0/3774134480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c009400 con 0x7f5738006ec0 2024-03-14T21:14:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581dff7640 1 -- 172.21.15.49:0/3982075046 learned_addr learned my addr 172.21.15.49:0/3982075046 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581d7f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581814e580 0x7f58180b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f5273fff640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f525c004b60 0x7f526c02a3b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5268035040 tx=0x7f5268001250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f91aaafa640 1 -- start start 2024-03-14T21:14:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581e7f8640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818145a80 0x7f58181e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a406f900 con 0x7f96a414a810 2024-03-14T21:14:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.577+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a406fec0 con 0x7f96a414a810 2024-03-14T21:14:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f226cff9640 1 -- 172.21.15.49:0/2353728913 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2264004e30 con 0x7f22700b7f50 2024-03-14T21:14:08.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f226f7fe640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f225c06a7a0 0x7f225c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f573ffff640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5738006ec0 0x7f573801d3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f573400b4b0 tx=0x7f573400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f91aaafa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91a40a4c70 0x7f91a4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f5270ff9640 1 -- 172.21.15.49:0/238146513 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268031630 con 0x7f525c004b60 2024-03-14T21:14:08.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f5270ff9640 1 -- 172.21.15.49:0/238146513 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268031790 con 0x7f525c004b60 2024-03-14T21:14:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f91aaafa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40bac30 0x7f91a4070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f91aaafa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a414fa40 0x7f91a406f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f5270ff9640 1 -- 172.21.15.49:0/238146513 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268031b40 con 0x7f525c004b60 2024-03-14T21:14:08.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581d7f6640 1 -- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58040047c0 msgr2=0x7f58181453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581d7f6640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58040047c0 0x7f58181453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f91aaafa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a409ffe0 con 0x7f91a40a4c70 2024-03-14T21:14:08.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f91aaafa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a4070fb0 con 0x7f91a414fa40 2024-03-14T21:14:08.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f91aaafa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a406f4a0 con 0x7f91a40bac30 2024-03-14T21:14:08.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581d7f6640 1 -- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818145a80 msgr2=0x7f58181e3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581d7f6640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818145a80 0x7f58181e3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f581d7f6640 1 -- 172.21.15.49:0/3982075046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581814e060 con 0x7f581814e580 2024-03-14T21:14:08.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f573cff9640 1 -- 172.21.15.49:0/3774134480 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734011830 con 0x7f5738006ec0 2024-03-14T21:14:08.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f581d7f6640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581814e580 0x7f58180b9dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5804004720 tx=0x7f5814004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57380d5030 con 0x7f5738006ec0 2024-03-14T21:14:08.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f573cff9640 1 -- 172.21.15.49:0/3774134480 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734011990 con 0x7f5738006ec0 2024-03-14T21:14:08.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573801db90 con 0x7f5738006ec0 2024-03-14T21:14:08.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f226f7fe640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f225c06a7a0 0x7f225c06cc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f226000b240 tx=0x7f226000b450 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c16a6e0 con 0x7f525c004b60 2024-03-14T21:14:08.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.578+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c028630 con 0x7f525c004b60 2024-03-14T21:14:08.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f91aa2f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a414fa40 0x7f91a406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f91aa2f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a414fa40 0x7f91a406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33932/0 (socket says 172.21.15.49:33932) 2024-03-14T21:14:08.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f91aa2f9640 1 -- 172.21.15.49:0/2182605761 learned_addr learned my addr 172.21.15.49:0/2182605761 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f91a9af8640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91a40a4c70 0x7f91a4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f91a92f7640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40bac30 0x7f91a4070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f91a9af8640 1 -- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40bac30 msgr2=0x7f91a4070a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f91a9af8640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40bac30 0x7f91a4070a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f91a9af8640 1 -- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a414fa40 msgr2=0x7f91a406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f91a9af8640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a414fa40 0x7f91a406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f91a9af8640 1 -- 172.21.15.49:0/2182605761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00094d0 con 0x7f91a40a4c70 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f91aa2f9640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a414fa40 0x7f91a406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:08.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c006860 con 0x7f525c004b60 2024-03-14T21:14:08.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a406d0a0 con 0x7f96a414a810 2024-03-14T21:14:08.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f91a9af8640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91a40a4c70 0x7f91a4071a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91a0033000 tx=0x7f91a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f573cff9640 1 -- 172.21.15.49:0/3774134480 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734011d40 con 0x7f5738006ec0 2024-03-14T21:14:08.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f580e7fc640 1 -- 172.21.15.49:0/3982075046 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814034c80 con 0x7f581814e580 2024-03-14T21:14:08.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.579+0000 7f580e7fc640 1 -- 172.21.15.49:0/3982075046 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814034de0 con 0x7f581814e580 2024-03-14T21:14:08.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f580e7fc640 1 -- 172.21.15.49:0/3982075046 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814004260 con 0x7f581814e580 2024-03-14T21:14:08.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f581eff9640 1 -- 172.21.15.49:0/3982075046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5818147fd0 con 0x7f581814e580 2024-03-14T21:14:08.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.580+0000 7f581eff9640 1 -- 172.21.15.49:0/3982075046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5818145f10 con 0x7f581814e580 2024-03-14T21:14:08.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.581+0000 7f919a7fc640 1 -- 172.21.15.49:0/2182605761 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0033a80 con 0x7f91a40a4c70 2024-03-14T21:14:08.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.581+0000 7f919a7fc640 1 -- 172.21.15.49:0/2182605761 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a0033c00 con 0x7f91a40a4c70 2024-03-14T21:14:08.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.581+0000 7f91aaafa640 1 -- 172.21.15.49:0/2182605761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a406fe70 con 0x7f91a40a4c70 2024-03-14T21:14:08.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.581+0000 7f91aaafa640 1 -- 172.21.15.49:0/2182605761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a406ee50 con 0x7f91a40a4c70 2024-03-14T21:14:08.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.581+0000 7f96a0ff9640 1 -- 172.21.15.49:0/2177679077 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f969c007070 con 0x7f96a414a810 2024-03-14T21:14:08.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.581+0000 7f573cff9640 1 -- 172.21.15.49:0/3774134480 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f573401a400 con 0x7f5738006ec0 2024-03-14T21:14:08.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f96a0ff9640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f968806a7f0 0x7f968806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f919a7fc640 1 -- 172.21.15.49:0/2182605761 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a003c400 con 0x7f91a40a4c70 2024-03-14T21:14:08.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5818147e30 con 0x7f581814e580 2024-03-14T21:14:08.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f96a0ff9640 1 -- 172.21.15.49:0/2177679077 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f969c088ae0 con 0x7f96a414a810 2024-03-14T21:14:08.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a40716e0 con 0x7f91a40a4c70 2024-03-14T21:14:08.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5720005250 con 0x7f5738006ec0 2024-03-14T21:14:08.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f573cff9640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f572406a7f0 0x7f572406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f573f7fe640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f572406a7f0 0x7f572406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f96a3fff640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f968806a7f0 0x7f968806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f580e7fc640 1 -- 172.21.15.49:0/3982075046 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5814003610 con 0x7f581814e580 2024-03-14T21:14:08.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.582+0000 7f573cff9640 1 -- 172.21.15.49:0/3774134480 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57340880a0 con 0x7f5738006ec0 2024-03-14T21:14:08.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f919a7fc640 1 -- 172.21.15.49:0/2182605761 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f91a003a040 con 0x7f91a40a4c70 2024-03-14T21:14:08.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f573f7fe640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f572406a7f0 0x7f572406ccb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f572c036000 tx=0x7f572c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f96a3fff640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f968806a7f0 0x7f968806ccb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9698006fd0 tx=0x7f9698008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f580e7fc640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f57fc06a7f0 0x7f57fc06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f580e7fc640 1 -- 172.21.15.49:0/3982075046 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58140aab20 con 0x7f581814e580 2024-03-14T21:14:08.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f581dff7640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f57fc06a7f0 0x7f57fc06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f919a7fc640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f918806a870 0x7f918806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f919a7fc640 1 -- 172.21.15.49:0/2182605761 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91a00aa6c0 con 0x7f91a40a4c70 2024-03-14T21:14:08.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.583+0000 7f91a92f7640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f918806a870 0x7f918806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.584+0000 7f581dff7640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f57fc06a7f0 0x7f57fc06ccb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58080017e0 tx=0x7f5808001430 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.584+0000 7f91a92f7640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f918806a870 0x7f918806cd30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9194006fd0 tx=0x7f9194008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.587+0000 7f62297fa640 1 -- 172.21.15.49:0/2757711833 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218039c20 con 0x7f62240072a0 2024-03-14T21:14:08.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.587+0000 7f47c9ffb640 1 -- 172.21.15.49:0/775509863 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f47cc076b40 con 0x7f47d4154420 2024-03-14T21:14:08.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.587+0000 7f62297fa640 1 -- 172.21.15.49:0/2757711833 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218039d80 con 0x7f62240072a0 2024-03-14T21:14:08.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.587+0000 7f62297fa640 1 -- 172.21.15.49:0/2757711833 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218043630 con 0x7f62240072a0 2024-03-14T21:14:08.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.587+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62240ca730 con 0x7f62240072a0 2024-03-14T21:14:08.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.587+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62240cc440 con 0x7f62240072a0 2024-03-14T21:14:08.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.588+0000 7f573cff9640 1 -- 172.21.15.49:0/3774134480 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f573408d060 con 0x7f5738006ec0 2024-03-14T21:14:08.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.588+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62240cc660 con 0x7f62240072a0 2024-03-14T21:14:08.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.588+0000 7f226cff9640 1 -- 172.21.15.49:0/2353728913 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f22640539f0 con 0x7f22700b7f50 2024-03-14T21:14:08.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.589+0000 7f62297fa640 1 -- 172.21.15.49:0/2757711833 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6218043790 con 0x7f62240072a0 2024-03-14T21:14:08.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.589+0000 7f62297fa640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f621006a7d0 0x7f621006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.589+0000 7f62297fa640 1 -- 172.21.15.49:0/2757711833 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62180ab6f0 con 0x7f62240072a0 2024-03-14T21:14:08.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.590+0000 7f6230a6a640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f621006a7d0 0x7f621006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.590+0000 7f5270ff9640 1 -- 172.21.15.49:0/238146513 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5268039030 con 0x7f525c004b60 2024-03-14T21:14:08.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.590+0000 7f6230a6a640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f621006a7d0 0x7f621006cc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f622000a8b0 tx=0x7f6220008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.590+0000 7f5270ff9640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f526006a820 0x7f526006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:08.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.590+0000 7f52737fe640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f526006a820 0x7f526006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:08.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.591+0000 7f52737fe640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f526006a820 0x7f526006cce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f526c167810 tx=0x7f5264008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:08.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.596+0000 7f580e7fc640 1 -- 172.21.15.49:0/3982075046 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5818147e30 con 0x7f581814e580 2024-03-14T21:14:08.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.598+0000 7f5270ff9640 1 -- 172.21.15.49:0/238146513 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f526807a010 con 0x7f525c004b60 2024-03-14T21:14:08.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.598+0000 7f96a0ff9640 1 -- 172.21.15.49:0/2177679077 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f969c054120 con 0x7f96a414a810 2024-03-14T21:14:08.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.599+0000 7f62297fa640 1 -- 172.21.15.49:0/2757711833 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6218076b10 con 0x7f62240072a0 2024-03-14T21:14:08.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.603+0000 7f5270ff9640 1 -- 172.21.15.49:0/238146513 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5268076350 con 0x7f525c004b60 2024-03-14T21:14:08.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.605+0000 7f919a7fc640 1 -- 172.21.15.49:0/2182605761 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f91a40716e0 con 0x7f91a40a4c70 2024-03-14T21:14:08.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.995+0000 7f22567fc640 1 -- 172.21.15.49:0/2353728913 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2270073e50 con 0x7f22700b7f50 2024-03-14T21:14:08.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.995+0000 7f226cff9640 1 -- 172.21.15.49:0/2353728913 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2264053b90 con 0x7f22700b7f50 2024-03-14T21:14:08.979 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:08.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.999+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f225c06a7a0 msgr2=0x7f225c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.999+0000 7f227694e640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f225c06a7a0 0x7f225c06cc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f226000b240 tx=0x7f226000b450 comp rx=0 tx=0).stop 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.999+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22700b7f50 msgr2=0x7f22700a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.999+0000 7f227694e640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22700b7f50 0x7f22700a04f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f226400c450 tx=0x7f226400c910 comp rx=0 tx=0).stop 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.999+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 shutdown_connections 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.999+0000 7f227694e640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f225c06a7a0 0x7f225c06cc60 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f226000b240 tx=0x7f226000b450 comp rx=0 tx=0).stop 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:08.999+0000 7f227694e640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22701480f0 0x7f227013e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.000+0000 7f227694e640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22701409a0 0x7f22700b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.000+0000 7f227694e640 1 --2- 172.21.15.49:0/2353728913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22700b7f50 0x7f22700a04f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.000+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 >> 172.21.15.49:0/2353728913 conn(0x7f22700b8b80 msgr2=0x7f22700b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.001+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 shutdown_connections 2024-03-14T21:14:08.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.001+0000 7f227694e640 1 -- 172.21.15.49:0/2353728913 wait complete. 2024-03-14T21:14:08.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.003+0000 7f47db9bb640 1 -- 172.21.15.49:0/775509863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f47b4005500 con 0x7f47d4154420 2024-03-14T21:14:08.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.003+0000 7f47c9ffb640 1 -- 172.21.15.49:0/775509863 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47cc039020 con 0x7f47d4154420 2024-03-14T21:14:08.987 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-03-14T21:14:08.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f57200057d0 con 0x7f5738006ec0 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 -- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f47b806a870 msgr2=0x7f47b806cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f47b806a870 0x7f47b806cd30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47d00016c0 tx=0x7f47d0009040 comp rx=0 tx=0).stop 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 -- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d4154420 msgr2=0x7f47d41e3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d4154420 0x7f47d41e3930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47d406af90 tx=0x7f47cc004ab0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 -- 172.21.15.49:0/775509863 shutdown_connections 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f47b806a870 0x7f47b806cd30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d4154420 0x7f47d41e3930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47d414b520 0x7f47d4149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 --2- 172.21.15.49:0/775509863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47d4145060 0x7f47d4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.007+0000 7f47bf7fe640 1 -- 172.21.15.49:0/775509863 >> 172.21.15.49:0/775509863 conn(0x7f47d4069ce0 msgr2=0x7f47d4142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.008+0000 7f573cff9640 1 -- 172.21.15.49:0/3774134480 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5734015090 con 0x7f5738006ec0 2024-03-14T21:14:08.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.008+0000 7f47bf7fe640 1 -- 172.21.15.49:0/775509863 shutdown_connections 2024-03-14T21:14:08.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.008+0000 7f47bf7fe640 1 -- 172.21.15.49:0/775509863 wait complete. 2024-03-14T21:14:08.992 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-03-14T21:14:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.011+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f572406a7f0 msgr2=0x7f572406ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.011+0000 7f574552c640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f572406a7f0 0x7f572406ccb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f572c036000 tx=0x7f572c008030 comp rx=0 tx=0).stop 2024-03-14T21:14:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.011+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5738006ec0 msgr2=0x7f573801d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:08.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.011+0000 7f574552c640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5738006ec0 0x7f573801d3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f573400b4b0 tx=0x7f573400b8b0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 shutdown_connections 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f572406a7f0 0x7f572406ccb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57380d53e0 0x7f57380d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57380072a0 0x7f57380d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 --2- 172.21.15.49:0/3774134480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5738006ec0 0x7f573801d3e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 >> 172.21.15.49:0/3774134480 conn(0x7f573803edf0 msgr2=0x7f5738040950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 shutdown_connections 2024-03-14T21:14:08.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.012+0000 7f574552c640 1 -- 172.21.15.49:0/3774134480 wait complete. 2024-03-14T21:14:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.018+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f526c16a160 con 0x7f525c004b60 2024-03-14T21:14:09.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.018+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5818147e30 con 0x7f581814e580 2024-03-14T21:14:09.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.019+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6224005290 con 0x7f62240072a0 2024-03-14T21:14:09.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-03-14T21:14:09.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.019+0000 7f5270ff9640 1 -- 172.21.15.49:0/238146513 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f526804b090 con 0x7f525c004b60 2024-03-14T21:14:09.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.019+0000 7f580e7fc640 1 -- 172.21.15.49:0/3982075046 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5818147e30 con 0x7f581814e580 2024-03-14T21:14:09.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.019+0000 7f62297fa640 1 -- 172.21.15.49:0/2757711833 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f621807a130 con 0x7f62240072a0 2024-03-14T21:14:09.004 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-03-14T21:14:09.004 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-03-14T21:14:09.004 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f526006a820 msgr2=0x7f526006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f526006a820 0x7f526006cce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f526c167810 tx=0x7f5264008040 comp rx=0 tx=0).stop 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f525c004b60 msgr2=0x7f526c02a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f525c004b60 0x7f526c02a3b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5268035040 tx=0x7f5268001250 comp rx=0 tx=0).stop 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 shutdown_connections 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f526006a820 0x7f526006cce0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f526c164110 0x7f526c028b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f526c02b050 0x7f526c163bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 --2- 172.21.15.49:0/238146513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f525c004b60 0x7f526c02a3b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 >> 172.21.15.49:0/238146513 conn(0x7f526c043220 msgr2=0x7f526c15bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 shutdown_connections 2024-03-14T21:14:09.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f527901f640 1 -- 172.21.15.49:0/238146513 wait complete. 2024-03-14T21:14:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f621006a7d0 msgr2=0x7f621006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f57fc06a7f0 msgr2=0x7f57fc06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f57fc06a7f0 0x7f57fc06ccb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58080017e0 tx=0x7f5808001430 comp rx=0 tx=0).stop 2024-03-14T21:14:09.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.022+0000 7f6231a6c640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f621006a7d0 0x7f621006cc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f622000a8b0 tx=0x7f6220008040 comp rx=0 tx=0).stop 2024-03-14T21:14:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581814e580 msgr2=0x7f58180b9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581814e580 0x7f58180b9dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5804004720 tx=0x7f5814004730 comp rx=0 tx=0).stop 2024-03-14T21:14:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62240072a0 msgr2=0x7f62240cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62240072a0 0x7f62240cfda0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6218000c00 tx=0x7f6218004730 comp rx=0 tx=0).stop 2024-03-14T21:14:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 shutdown_connections 2024-03-14T21:14:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 shutdown_connections 2024-03-14T21:14:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f57fc06a7f0 0x7f57fc06ccb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818145a80 0x7f58181e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581814e580 0x7f58180b9dc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 --2- 172.21.15.49:0/3982075046 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58040047c0 0x7f58181453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 >> 172.21.15.49:0/3982075046 conn(0x7f58180a1e20 msgr2=0x7f5818141db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f621006a7d0 0x7f621006cc90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62240d53e0 0x7f62240c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62240072a0 0x7f62240cfda0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 --2- 172.21.15.49:0/2757711833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6224006ec0 0x7f62240d8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 >> 172.21.15.49:0/2757711833 conn(0x7f622403edf0 msgr2=0x7f62240d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 shutdown_connections 2024-03-14T21:14:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 shutdown_connections 2024-03-14T21:14:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.023+0000 7f6231a6c640 1 -- 172.21.15.49:0/2757711833 wait complete. 2024-03-14T21:14:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.024+0000 7f57fbfff640 1 -- 172.21.15.49:0/3982075046 wait complete. 2024-03-14T21:14:09.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-03-14T21:14:09.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.026+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f91a4066490 con 0x7f91a40a4c70 2024-03-14T21:14:09.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.027+0000 7f919a7fc640 1 -- 172.21.15.49:0/2182605761 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91a4066490 con 0x7f91a40a4c70 2024-03-14T21:14:09.013 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.028+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f96a40700b0 con 0x7f96a414a810 2024-03-14T21:14:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.029+0000 7f96a0ff9640 1 -- 172.21.15.49:0/2177679077 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f969c017020 con 0x7f96a414a810 2024-03-14T21:14:09.014 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f918806a870 msgr2=0x7f918806cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f918806a870 0x7f918806cd30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9194006fd0 tx=0x7f9194008040 comp rx=0 tx=0).stop 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91a40a4c70 msgr2=0x7f91a4071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91a40a4c70 0x7f91a4071a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91a0033000 tx=0x7f91a00046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 shutdown_connections 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f918806a870 0x7f918806cd30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91a414fa40 0x7f91a406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91a40bac30 0x7f91a4070a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.030+0000 7f9187fff640 1 --2- 172.21.15.49:0/2182605761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91a40a4c70 0x7f91a4071a30 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91a0033000 tx=0x7f91a00046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.031+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 >> 172.21.15.49:0/2182605761 conn(0x7f91a40bcac0 msgr2=0x7f91a40bdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.031+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 shutdown_connections 2024-03-14T21:14:09.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.031+0000 7f9187fff640 1 -- 172.21.15.49:0/2182605761 wait complete. 2024-03-14T21:14:09.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-03-14T21:14:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f968806a7f0 msgr2=0x7f968806ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f968806a7f0 0x7f968806ccb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9698006fd0 tx=0x7f9698008040 comp rx=0 tx=0).stop 2024-03-14T21:14:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a414a810 msgr2=0x7f96a4098bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a414a810 0x7f96a4098bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f969c00cb20 tx=0x7f969c00cf20 comp rx=0 tx=0).stop 2024-03-14T21:14:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 shutdown_connections 2024-03-14T21:14:09.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f968806a7f0 0x7f968806ccb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96a414a810 0x7f96a4098bc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96a4144b70 0x7f96a40703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 --2- 172.21.15.49:0/2177679077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a409ece0 0x7f96a4071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 >> 172.21.15.49:0/2177679077 conn(0x7f96a40b8b80 msgr2=0x7f96a4142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 shutdown_connections 2024-03-14T21:14:09.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:09.033+0000 7f96aa804640 1 -- 172.21.15.49:0/2177679077 wait complete. 2024-03-14T21:14:09.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-03-14T21:14:09.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-03-14T21:14:09.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-03-14T21:14:09.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-03-14T21:14:09.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-03-14T21:14:10.004 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-14T21:14:10.014 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-14T21:14:10.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-14T21:14:10.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-14T21:14:10.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-14T21:14:10.027 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-14T21:14:10.029 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-14T21:14:10.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-14T21:14:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.200+0000 7fa8c83e7640 1 Processor -- start 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.200+0000 7fa8c83e7640 1 -- start start 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c83e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b7e10 0x7fa8c01470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c83e7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8c00b11b0 0x7fa8c01407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c83e7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c006b820 con 0x7fa8c0147610 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c83e7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c01469c0 con 0x7fa8c00b11b0 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c83e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00b1bd0 con 0x7fa8c00b7e10 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c5982640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8c00b11b0 0x7fa8c01407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c5982640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa8c00b11b0 0x7fa8c01407d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37632/0 (socket says 172.21.15.49:37632) 2024-03-14T21:14:10.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.202+0000 7fa8c5982640 1 -- 172.21.15.49:0/3317080396 learned_addr learned my addr 172.21.15.49:0/3317080396 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.203+0000 7fa8c5982640 1 -- 172.21.15.49:0/3317080396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b7e10 msgr2=0x7fa8c01470d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T21:14:10.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.203+0000 7fa8c5982640 1 --2- 172.21.15.49:0/3317080396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b7e10 0x7fa8c01470d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.203+0000 7fa8c5982640 1 -- 172.21.15.49:0/3317080396 >> v1:172.21.15.97:6789/0 conn(0x7fa8c0147610 legacy=0x7fa8c0149f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T21:14:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.203+0000 7fa8c5982640 1 -- 172.21.15.49:0/3317080396 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00b1eb0 con 0x7fa8c00b11b0 2024-03-14T21:14:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.204+0000 7fa8c5982640 1 --2- 172.21.15.49:0/3317080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b11b0 0x7fa8c01407d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800d370 tx=0x7fa8b802e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae03975e23819adb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.206+0000 7fa8c5181640 1 -- 172.21.15.49:0/3317080396 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8038600 con 0x7fa8c00b11b0 2024-03-14T21:14:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.206+0000 7fa8c5181640 1 -- 172.21.15.49:0/3317080396 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b8032070 con 0x7fa8c00b11b0 2024-03-14T21:14:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.207+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3317080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b11b0 msgr2=0x7fa8c01407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.207+0000 7fa8c83e7640 1 --2- 172.21.15.49:0/3317080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b11b0 0x7fa8c01407d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800d370 tx=0x7fa8b802e2a0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.214+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3317080396 shutdown_connections 2024-03-14T21:14:10.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.214+0000 7fa8c83e7640 1 --2- 172.21.15.49:0/3317080396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b11b0 0x7fa8c01407d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.214+0000 7fa8c83e7640 1 --2- 172.21.15.49:0/3317080396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b7e10 0x7fa8c01470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.214+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3317080396 >> 172.21.15.49:0/3317080396 conn(0x7fa8c0069ce0 msgr2=0x7fa8c006a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.214+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3317080396 shutdown_connections 2024-03-14T21:14:10.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.214+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3317080396 wait complete. 2024-03-14T21:14:10.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.216+0000 7fa8c83e7640 1 Processor -- start 2024-03-14T21:14:10.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.216+0000 7fa8c83e7640 1 -- start start 2024-03-14T21:14:10.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.217+0000 7fa8c83e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b11b0 0x7fa8c01cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.217+0000 7fa8c83e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b7e10 0x7fa8c009ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c6183640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b11b0 0x7fa8c01cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c6183640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b11b0 0x7fa8c01cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46592/0 (socket says 172.21.15.49:46592) 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c6183640 1 -- 172.21.15.49:0/3984006867 learned_addr learned my addr 172.21.15.49:0/3984006867 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c5982640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b7e10 0x7fa8c009ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c83e7640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c0147610 0x7fa8c00a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c01f0930 con 0x7fa8c00b7e10 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c01efd20 con 0x7fa8c0147610 2024-03-14T21:14:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.218+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c01f0000 con 0x7fa8c00b11b0 2024-03-14T21:14:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.219+0000 7fa8c5982640 1 -- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b11b0 msgr2=0x7fa8c01cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.219+0000 7fa8c5982640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b11b0 0x7fa8c01cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.219+0000 7fa8c5982640 1 -- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c0147610 msgr2=0x7fa8c00a0e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T21:14:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.219+0000 7fa8c5982640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c0147610 0x7fa8c00a0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.219+0000 7fa8c5982640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b800d050 con 0x7fa8c00b7e10 2024-03-14T21:14:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.220+0000 7fa8c5982640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b7e10 0x7fa8c009ce50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8036000 tx=0x7fa8b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.220+0000 7fa8b6ffd640 1 -- 172.21.15.49:0/3984006867 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8008240 con 0x7fa8c00b7e10 2024-03-14T21:14:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.221+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c01f02e0 con 0x7fa8c00b7e10 2024-03-14T21:14:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.221+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c00b6780 con 0x7fa8c00b7e10 2024-03-14T21:14:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.224+0000 7fa8b6ffd640 1 -- 172.21.15.49:0/3984006867 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80083c0 con 0x7fa8c00b7e10 2024-03-14T21:14:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.225+0000 7fa8b6ffd640 1 -- 172.21.15.49:0/3984006867 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8008770 con 0x7fa8c00b7e10 2024-03-14T21:14:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.225+0000 7fa8b6ffd640 1 -- 172.21.15.49:0/3984006867 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa8b804c460 con 0x7fa8c00b7e10 2024-03-14T21:14:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.225+0000 7fa8b6ffd640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa8a406a920 0x7fa8a406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.226+0000 7fa8b6ffd640 1 -- 172.21.15.49:0/3984006867 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8b80aba10 con 0x7fa8c00b7e10 2024-03-14T21:14:10.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.226+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c01f0aa0 con 0x7fa8c00b7e10 2024-03-14T21:14:10.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.232+0000 7fa8b6ffd640 1 -- 172.21.15.49:0/3984006867 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa8b8076d80 con 0x7fa8c00b7e10 2024-03-14T21:14:10.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.235+0000 7fa8c6183640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa8a406a920 0x7fa8a406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.241+0000 7fa8c6183640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa8a406a920 0x7fa8a406cde0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009f2c0 tx=0x7fa8bc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.246+0000 7f0fb15ff640 1 Processor -- start 2024-03-14T21:14:10.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.246+0000 7f0fb15ff640 1 -- start start 2024-03-14T21:14:10.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.247+0000 7f0fb15ff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fa4042e40 0x7f0fa4008e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.247+0000 7f0fb15ff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0fa40ce130 0x7f0fa40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.247+0000 7f0fb15ff640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa403de00 con 0x7f0fa400b3d0 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.247+0000 7f0fb15ff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa4009390 con 0x7f0fa40ce130 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.247+0000 7f0fb15ff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa403e6a0 con 0x7f0fa4042e40 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.247+0000 7f0fabfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0fa400b3d0 0x7f0fa40cb1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:36942/0 (socket says 172.21.15.49:36942) 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.247+0000 7f0fabfff640 1 -- 172.21.15.49:0/2923814650 learned_addr learned my addr 172.21.15.49:0/2923814650 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966965232 0 0) 0x7f0fa403de00 con 0x7f0fa400b3d0 2024-03-14T21:14:10.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f9c003520 con 0x7f0fa400b3d0 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0fb0dfe640 1 --2- 172.21.15.49:0/2923814650 >> v2:172.21.15.49:3300/0 conn(0x7f0fa40ce130 0x7f0fa40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950795808 0 0) 0x7f0f9c003520 con 0x7f0fa400b3d0 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 >> v2:172.21.15.97:3300/0 conn(0x7f0fa4042e40 msgr2=0x7f0fa4008e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0fab7fe640 1 --2- 172.21.15.49:0/2923814650 >> v2:172.21.15.97:3300/0 conn(0x7f0fa4042e40 0x7f0fa4008e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 --2- 172.21.15.49:0/2923814650 >> v2:172.21.15.97:3300/0 conn(0x7f0fa4042e40 0x7f0fa4008e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 >> v2:172.21.15.49:3300/0 conn(0x7f0fa40ce130 msgr2=0x7f0fa40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 --2- 172.21.15.49:0/2923814650 >> v2:172.21.15.49:3300/0 conn(0x7f0fa40ce130 0x7f0fa40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.248+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40ceaa0 con 0x7f0fa400b3d0 2024-03-14T21:14:10.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f0fa0001670 con 0x7f0fa400b3d0 2024-03-14T21:14:10.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 >> v1:172.21.15.49:6789/0 conn(0x7f0fa400b3d0 legacy=0x7f0fa40cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c0039c0 0x7f0f9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0fabfff640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c0039c0 0x7f0f9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0f9c0047c0 0x7f0f9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f9c009970 0x7f0f9c00bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa4009390 con 0x7f0f9c0039c0 2024-03-14T21:14:10.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa403e6a0 con 0x7f0f9c009970 2024-03-14T21:14:10.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c00c3d0 con 0x7f0f9c0047c0 2024-03-14T21:14:10.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0fb0dfe640 1 --2- 172.21.15.49:0/2923814650 >> v2:172.21.15.49:3300/0 conn(0x7f0fa40ce130 0x7f0fa40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:10.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.249+0000 7f0fab7fe640 1 --2- 172.21.15.49:0/2923814650 >> v2:172.21.15.97:3300/0 conn(0x7f0fa4042e40 0x7f0fa4008e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:10.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.250+0000 7f0fb0dfe640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f9c009970 0x7f0f9c00bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.250+0000 7f0fab7fe640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0f9c0047c0 0x7f0f9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.250+0000 7f0fb0dfe640 1 -- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0f9c0047c0 msgr2=0x7f0f9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.250+0000 7f0fb0dfe640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0f9c0047c0 0x7f0f9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.250+0000 7f0fb0dfe640 1 -- 172.21.15.49:0/2923814650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c0039c0 msgr2=0x7f0f9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.250+0000 7f0fb0dfe640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c0039c0 0x7f0f9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.250+0000 7f0fb0dfe640 1 -- 172.21.15.49:0/2923814650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c003520 con 0x7f0f9c009970 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.251+0000 7f0fb0dfe640 1 -- 172.21.15.49:0/2923814650 reap_dead start 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.252+0000 7f0fb0dfe640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f9c009970 0x7f0f9c00bd60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fac04f2a0 tx=0x7f0f9c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1cf8d197b368fa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.252+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0a7040 con 0x7f0f9c009970 2024-03-14T21:14:10.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.252+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac0ab6c0 con 0x7f0f9c009970 2024-03-14T21:14:10.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.252+0000 7f0faaffd640 1 -- 172.21.15.49:0/2923814650 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0aba30 con 0x7f0f9c009970 2024-03-14T21:14:10.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.252+0000 7f0fabfff640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c0039c0 0x7f0f9c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:10.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7fe5f88b9640 1 Processor -- start 2024-03-14T21:14:10.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7fe5f88b9640 1 -- start start 2024-03-14T21:14:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f9c009970 msgr2=0x7f0f9c00bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f9c009970 0x7f0f9c00bd60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fac04f2a0 tx=0x7f0f9c0057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2923814650 shutdown_connections 2024-03-14T21:14:10.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7fb1be0ef640 1 Processor -- start 2024-03-14T21:14:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2923814650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f9c009970 0x7f0f9c00bd60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2923814650 >> 172.21.15.49:0/2923814650 conn(0x7f0fa400a550 msgr2=0x7f0fa400a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7fb1be0ef640 1 -- start start 2024-03-14T21:14:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.253+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2923814650 shutdown_connections 2024-03-14T21:14:10.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f88b9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f406f170 0x7fe5f414c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f88b9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5f41485f0 0x7fe5f4148df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f88b9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f40b4460 con 0x7fe5f40b4820 2024-03-14T21:14:10.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f88b9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f4148a80 con 0x7fe5f41485f0 2024-03-14T21:14:10.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f88b9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f406f5b0 con 0x7fe5f406f170 2024-03-14T21:14:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fb1be0ef640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b0006eb0 0x7fb1b003c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fb1be0ef640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b003a5c0 con 0x7fb1b00d53d0 2024-03-14T21:14:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fb1be0ef640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b003a8a0 con 0x7fb1b0007290 2024-03-14T21:14:10.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fb1be0ef640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b003ab80 con 0x7fb1b0006eb0 2024-03-14T21:14:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5f41485f0 0x7fe5f4148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5f41485f0 0x7fe5f4148df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37666/0 (socket says 172.21.15.49:37666) 2024-03-14T21:14:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7fe5f3fff640 1 -- 172.21.15.49:0/1738613083 learned_addr learned my addr 172.21.15.49:0/1738613083 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fb1bd0ed640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b0006eb0 0x7fb1b003c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fb1bd8ee640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb1b0007290 0x7fb1b00dfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47812/0 (socket says 172.21.15.49:47812) 2024-03-14T21:14:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fb1bd0ed640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b0006eb0 0x7fb1b003c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46622/0 (socket says 172.21.15.49:46622) 2024-03-14T21:14:10.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fb1bd8ee640 1 -- 172.21.15.49:0/4004248091 learned_addr learned my addr 172.21.15.49:0/4004248091 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7f3658bca640 1 Processor -- start 2024-03-14T21:14:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7f3658bca640 1 -- start start 2024-03-14T21:14:10.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2923814650 wait complete. 2024-03-14T21:14:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.254+0000 7f16db59e640 1 Processor -- start 2024-03-14T21:14:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7f16db59e640 1 -- start start 2024-03-14T21:14:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fb1bd0ed640 1 -- 172.21.15.49:0/4004248091 >> v1:172.21.15.97:6789/0 conn(0x7fb1b0007290 legacy=0x7fb1b00dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fb1bd0ed640 1 -- 172.21.15.49:0/4004248091 >> v1:172.21.15.49:6789/0 conn(0x7fb1b00d53d0 legacy=0x7fb1b003e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fb1bd0ed640 1 -- 172.21.15.49:0/4004248091 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b003e160 con 0x7fb1b0006eb0 2024-03-14T21:14:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f3658bca640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c0418f0 0x7f364c03f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f3658bca640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f364c03f870 0x7f364c03fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f16db59e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f16dc06d640 0x7f16dc071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f16db59e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16dc0b7e50 con 0x7f16dc0b9590 2024-03-14T21:14:10.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f16db59e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16dc0ba770 con 0x7f16dc14f330 2024-03-14T21:14:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f16db59e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16dc06fdc0 con 0x7f16dc06d640 2024-03-14T21:14:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7fb1bd0ed640 1 --2- 172.21.15.49:0/4004248091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b0006eb0 0x7fb1b003c420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000b60 tx=0x7fb1b80a5440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd3cb51cc46db9f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fe5f3fff640 1 -- 172.21.15.49:0/1738613083 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f406f170 msgr2=0x7fe5f414c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f3658bca640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c03dc70 con 0x7f364c03cd00 2024-03-14T21:14:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f3658bca640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c02bea0 con 0x7f364c03f870 2024-03-14T21:14:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f3658bca640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c03d420 con 0x7f364c0418f0 2024-03-14T21:14:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f16dad9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f16dc14f330 0x7f16dc159560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47816/0 (socket says 172.21.15.49:47816) 2024-03-14T21:14:10.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f16dad9d640 1 -- 172.21.15.49:0/1254083697 learned_addr learned my addr 172.21.15.49:0/1254083697 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:10.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f0fb15ff640 1 Processor -- start 2024-03-14T21:14:10.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7fb1affff640 1 -- 172.21.15.49:0/4004248091 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80aa7d0 con 0x7fb1b0006eb0 2024-03-14T21:14:10.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7fb1affff640 1 -- 172.21.15.49:0/4004248091 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b80a8050 con 0x7fb1b0006eb0 2024-03-14T21:14:10.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.256+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c0418f0 0x7f364c03f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f36537fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f364c03cd00 0x7f364c0411e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47818/0 (socket says 172.21.15.49:47818) 2024-03-14T21:14:10.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f36537fe640 1 -- 172.21.15.49:0/1363283251 learned_addr learned my addr 172.21.15.49:0/1363283251 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:10.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c0418f0 0x7f364c03f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46634/0 (socket says 172.21.15.49:46634) 2024-03-14T21:14:10.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16da59c640 1 --2- 172.21.15.49:0/1254083697 >> v2:172.21.15.49:3300/0 conn(0x7f16dc06d640 0x7f16dc071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f3653fff640 1 --2- 172.21.15.49:0/1363283251 >> v2:172.21.15.49:3300/0 conn(0x7f364c03f870 0x7f364c03fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162527781 0 0) 0x7f16dc0ba770 con 0x7f16dc14f330 2024-03-14T21:14:10.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c4003520 con 0x7f16dc14f330 2024-03-14T21:14:10.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858432004 0 0) 0x7f16dc0b7e50 con 0x7f16dc0b9590 2024-03-14T21:14:10.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16dc0ba770 con 0x7f16dc0b9590 2024-03-14T21:14:10.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fe5f3fff640 1 --2- 172.21.15.49:0/1738613083 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f406f170 0x7fe5f414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fe5f3fff640 1 -- 172.21.15.49:0/1738613083 >> v1:172.21.15.97:6789/0 conn(0x7fe5f40b4820 legacy=0x7fe5f40b4c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T21:14:10.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.255+0000 7fe5f3fff640 1 -- 172.21.15.49:0/1738613083 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f414bc60 con 0x7fe5f41485f0 2024-03-14T21:14:10.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f3652ffd640 1 -- 172.21.15.49:0/1363283251 >> v2:172.21.15.49:3300/0 conn(0x7f364c03f870 msgr2=0x7f364c03fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f3652ffd640 1 --2- 172.21.15.49:0/1363283251 >> v2:172.21.15.49:3300/0 conn(0x7f364c03f870 0x7f364c03fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f3652ffd640 1 -- 172.21.15.49:0/1363283251 >> v1:172.21.15.97:6789/0 conn(0x7f364c03cd00 legacy=0x7f364c0411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f3652ffd640 1 -- 172.21.15.49:0/1363283251 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c040220 con 0x7f364c0418f0 2024-03-14T21:14:10.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7fe5f3fff640 1 --2- 172.21.15.49:0/1738613083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f41485f0 0x7fe5f4148df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b1a0 tx=0x7fe5ec02c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f81a60dd3aa9e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7fb1be0ef640 1 -- 172.21.15.49:0/4004248091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b0006eb0 msgr2=0x7fb1b003c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/4004248091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b0006eb0 0x7fb1b003c420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000b60 tx=0x7fb1b80a5440 comp rx=0 tx=0).stop 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461489998 0 0) 0x7f16c4003520 con 0x7f16dc14f330 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 >> v2:172.21.15.49:3300/0 conn(0x7f16dc06d640 msgr2=0x7f16dc071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 --2- 172.21.15.49:0/1254083697 >> v2:172.21.15.49:3300/0 conn(0x7f16dc06d640 0x7f16dc071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 >> v1:172.21.15.49:6789/0 conn(0x7f16dc0b9590 legacy=0x7f16dc14ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc06ffd0 con 0x7f16dc14f330 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f16d0001670 con 0x7f16dc14f330 2024-03-14T21:14:10.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 >> v1:172.21.15.97:6789/0 conn(0x7f16dc14f330 legacy=0x7f16dc159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16c4003b50 0x7f16c4003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f16d959a640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16d959a640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c40051d0 0x7f16c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc06fdc0 con 0x7f16c40051d0 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c400c100 con 0x7f16c40047c0 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c400c2c0 con 0x7f16c4003b50 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7f0fb15ff640 1 -- start start 2024-03-14T21:14:10.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.257+0000 7fcd44957640 1 Processor -- start 2024-03-14T21:14:10.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7fcd44957640 1 -- start start 2024-03-14T21:14:10.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16da59c640 1 --2- 172.21.15.49:0/1254083697 >> v2:172.21.15.49:3300/0 conn(0x7f16dc06d640 0x7f16dc071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:10.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f412d5fb640 1 Processor -- start 2024-03-14T21:14:10.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7fb1be0ef640 1 -- 172.21.15.49:0/4004248091 shutdown_connections 2024-03-14T21:14:10.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/4004248091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b0006eb0 0x7fb1b003c420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16da59c640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16c4003b50 0x7f16c4003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16dad9d640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c40051d0 0x7f16c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7fb1be0ef640 1 -- 172.21.15.49:0/4004248091 >> 172.21.15.49:0/4004248091 conn(0x7fb1b003ede0 msgr2=0x7fb1b003f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fb15ff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c009970 0x7f0fa40cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fb15ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa40cc540 0x7f0fa416e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fb15ff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fa416ee20 0x7f0fa40d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fb15ff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa40d1f90 con 0x7f0f9c009970 2024-03-14T21:14:10.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fb15ff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa416f2a0 con 0x7f0fa40cc540 2024-03-14T21:14:10.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fb15ff640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa40cc9a0 con 0x7f0fa416ee20 2024-03-14T21:14:10.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c009970 0x7f0fa40cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c009970 0x7f0fa40cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37706/0 (socket says 172.21.15.49:37706) 2024-03-14T21:14:10.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fab7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa40cc540 0x7f0fa416e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f0fabfff640 1 -- 172.21.15.49:0/2353116137 learned_addr learned my addr 172.21.15.49:0/2353116137 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd44957640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd380ccee0 0x7fcd38007ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd44957640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcd380cd2c0 0x7fcd380cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fe5f27fc640 1 -- 172.21.15.49:0/1738613083 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec02cb50 con 0x7fe5f41485f0 2024-03-14T21:14:10.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fe5f27fc640 1 -- 172.21.15.49:0/1738613083 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec02ccb0 con 0x7fe5f41485f0 2024-03-14T21:14:10.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fe5f27fc640 1 -- 172.21.15.49:0/1738613083 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec034610 con 0x7fe5f41485f0 2024-03-14T21:14:10.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16d9d9b640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f16da59c640 1 -- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16c40047c0 msgr2=0x7f16c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f16da59c640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f16da59c640 1 -- 172.21.15.49:0/1254083697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c40051d0 msgr2=0x7f16c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f16da59c640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c40051d0 0x7f16c400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f16da59c640 1 -- 172.21.15.49:0/1254083697 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc06ffd0 con 0x7f16c4003b50 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f16d9d9b640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16c40047c0 0x7f16c4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f16dad9d640 1 -- 172.21.15.49:0/1254083697 reap_dead start 2024-03-14T21:14:10.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f0fab7fe640 1 -- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fa416ee20 msgr2=0x7f0fa40d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.258+0000 7f412d5fb640 1 -- start start 2024-03-14T21:14:10.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f412d5fb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4128152c60 0x7f41280b8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f412d5fb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412806bf30 con 0x7f41280b3080 2024-03-14T21:14:10.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f412d5fb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280b12e0 con 0x7f412814c520 2024-03-14T21:14:10.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f3652ffd640 1 --2- 172.21.15.49:0/1363283251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c0418f0 0x7f364c03f330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36480097a0 tx=0x7f364802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ebc7b2535b8c67 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f412d5fb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41280b1580 con 0x7f4128152c60 2024-03-14T21:14:10.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f0fab7fe640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fa416ee20 0x7f0fa40d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f0fab7fe640 1 -- 172.21.15.49:0/2353116137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c009970 msgr2=0x7f0fa40cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f0fab7fe640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c009970 0x7f0fa40cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f0fab7fe640 1 -- 172.21.15.49:0/2353116137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40cf670 con 0x7f0fa40cc540 2024-03-14T21:14:10.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f4126ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f412814c520 0x7f412806b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:47820/0 (socket says 172.21.15.49:47820) 2024-03-14T21:14:10.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7f4126ffd640 1 -- 172.21.15.49:0/206642954 learned_addr learned my addr 172.21.15.49:0/206642954 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:10.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fb1be0ef640 1 -- 172.21.15.49:0/4004248091 shutdown_connections 2024-03-14T21:14:10.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f41277fe640 1 --2- 172.21.15.49:0/206642954 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4128152c60 0x7f41280b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd3f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd380ccee0 0x7fcd38007ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd3f7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd380ccee0 0x7fcd38007ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33982/0 (socket says 172.21.15.49:33982) 2024-03-14T21:14:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/1845949148 learned_addr learned my addr 172.21.15.49:0/1845949148 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3803f040 con 0x7fcd38036310 2024-03-14T21:14:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3803edd0 con 0x7fcd380cd2c0 2024-03-14T21:14:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3803d600 con 0x7fcd380ccee0 2024-03-14T21:14:10.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.259+0000 7fcd3effd640 1 --2- 172.21.15.49:0/1845949148 >> v2:172.21.15.49:3300/0 conn(0x7fcd380cd2c0 0x7fcd380cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7fb1be0ef640 1 -- 172.21.15.49:0/4004248091 wait complete. 2024-03-14T21:14:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/1845949148 >> v2:172.21.15.49:3300/0 conn(0x7fcd380cd2c0 msgr2=0x7fcd380cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7fcd3f7fe640 1 --2- 172.21.15.49:0/1845949148 >> v2:172.21.15.49:3300/0 conn(0x7fcd380cd2c0 0x7fcd380cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/1845949148 >> v1:172.21.15.49:6789/0 conn(0x7fcd38036310 legacy=0x7fcd380d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f4125ffb640 1 -- 172.21.15.49:0/206642954 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148019170 0 0) 0x7f41280b12e0 con 0x7f412814c520 2024-03-14T21:14:10.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f0fab7fe640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa40cc540 0x7f0fa416e8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40d2490 tx=0x7f0f9800fbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/1845949148 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3803d810 con 0x7fcd380ccee0 2024-03-14T21:14:10.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f36527fc640 1 -- 172.21.15.49:0/1363283251 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648030610 con 0x7f364c0418f0 2024-03-14T21:14:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f36527fc640 1 -- 172.21.15.49:0/1363283251 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648030770 con 0x7f364c0418f0 2024-03-14T21:14:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f0fa8ff9640 1 -- 172.21.15.49:0/2353116137 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800d970 con 0x7f0fa40cc540 2024-03-14T21:14:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f0fa8ff9640 1 -- 172.21.15.49:0/2353116137 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9800c040 con 0x7f0fa40cc540 2024-03-14T21:14:10.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f0fa8ff9640 1 -- 172.21.15.49:0/2353116137 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800dcd0 con 0x7f0fa40cc540 2024-03-14T21:14:10.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7fe5f88b9640 1 -- 172.21.15.49:0/1738613083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f41485f0 msgr2=0x7fe5f4148df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/1738613083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f41485f0 0x7fe5f4148df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b1a0 tx=0x7fe5ec02c1b0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fe5f88b9640 1 -- 172.21.15.49:0/1738613083 shutdown_connections 2024-03-14T21:14:10.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/1738613083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f41485f0 0x7fe5f4148df0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa40cf950 con 0x7f0fa40cc540 2024-03-14T21:14:10.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa40d2200 con 0x7f0fa40cc540 2024-03-14T21:14:10.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/1738613083 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f406f170 0x7fe5f414c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fe5f88b9640 1 -- 172.21.15.49:0/1738613083 >> 172.21.15.49:0/1738613083 conn(0x7fe5f4070f20 msgr2=0x7fe5f4071300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f41277fe640 1 -- 172.21.15.49:0/206642954 >> v1:172.21.15.97:6789/0 conn(0x7f412814c520 legacy=0x7f412806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f41277fe640 1 -- 172.21.15.49:0/206642954 >> v1:172.21.15.49:6789/0 conn(0x7f41280b3080 legacy=0x7f4128152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.260+0000 7f41277fe640 1 -- 172.21.15.49:0/206642954 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41280b1860 con 0x7f4128152c60 2024-03-14T21:14:10.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f3658bca640 1 -- 172.21.15.49:0/1363283251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c0418f0 msgr2=0x7f364c03f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f16da59c640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16c4003b50 0x7f16c4003f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16cc006fd0 tx=0x7f16c40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99bfcaf1964ca78f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f36527fc640 1 -- 172.21.15.49:0/1363283251 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648030610 con 0x7f364c0418f0 2024-03-14T21:14:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f3658bca640 1 --2- 172.21.15.49:0/1363283251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c0418f0 0x7f364c03f330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36480097a0 tx=0x7f364802b960 comp rx=0 tx=0).stop 2024-03-14T21:14:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fe5f88b9640 1 -- 172.21.15.49:0/1738613083 shutdown_connections 2024-03-14T21:14:10.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc02d040 con 0x7f16c4003b50 2024-03-14T21:14:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc004230 con 0x7f16c4003b50 2024-03-14T21:14:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f16d959a640 1 -- 172.21.15.49:0/1254083697 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc003610 con 0x7f16c4003b50 2024-03-14T21:14:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fb1be0ef640 1 Processor -- start 2024-03-14T21:14:10.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fe5f88b9640 1 -- 172.21.15.49:0/1738613083 wait complete. 2024-03-14T21:14:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f41277fe640 1 --2- 172.21.15.49:0/206642954 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4128152c60 0x7f41280b8160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41100097d0 tx=0x7f411002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1191783f84959487 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f3658bca640 1 -- 172.21.15.49:0/1363283251 shutdown_connections 2024-03-14T21:14:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f3658bca640 1 --2- 172.21.15.49:0/1363283251 >> v2:172.21.15.49:3300/0 conn(0x7f364c03f870 0x7f364c03fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f3658bca640 1 --2- 172.21.15.49:0/1363283251 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c0418f0 0x7f364c03f330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7f3658bca640 1 -- 172.21.15.49:0/1363283251 >> 172.21.15.49:0/1363283251 conn(0x7f364c046970 msgr2=0x7f364c046d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7fcd3f7fe640 1 --2- 172.21.15.49:0/1845949148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380ccee0 0x7fcd38007ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800d090 tx=0x7fcd340341e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a300bed67feba23 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7f3658bca640 1 -- 172.21.15.49:0/1363283251 shutdown_connections 2024-03-14T21:14:10.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7f4125ffb640 1 -- 172.21.15.49:0/206642954 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110030610 con 0x7f4128152c60 2024-03-14T21:14:10.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7f4125ffb640 1 -- 172.21.15.49:0/206642954 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4110030770 con 0x7f4128152c60 2024-03-14T21:14:10.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.261+0000 7fb1be0ef640 1 -- start start 2024-03-14T21:14:10.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7fb1be0ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1b0006eb0 0x7fb1b00d2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7f4125ffb640 1 -- 172.21.15.49:0/206642954 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110030b40 con 0x7f4128152c60 2024-03-14T21:14:10.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1be0ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b0007290 0x7fb1b00376f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7f16db59e640 1 -- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16c4003b50 msgr2=0x7f16c4003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7f16db59e640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16c4003b50 0x7f16c4003f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16cc006fd0 tx=0x7f16c40057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f16db59e640 1 -- 172.21.15.49:0/1254083697 shutdown_connections 2024-03-14T21:14:10.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f16db59e640 1 --2- 172.21.15.49:0/1254083697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16c4003b50 0x7f16c4003f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f16db59e640 1 -- 172.21.15.49:0/1254083697 >> 172.21.15.49:0/1254083697 conn(0x7f16dc0b9a90 msgr2=0x7f16dc0b9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1be0ef640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b00d53d0 0x7fb1b0035150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7fcd3e7fc640 1 -- 172.21.15.49:0/1845949148 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34012ae0 con 0x7fcd380ccee0 2024-03-14T21:14:10.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7fcd3e7fc640 1 -- 172.21.15.49:0/1845949148 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34012c40 con 0x7fcd380ccee0 2024-03-14T21:14:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7fcd3e7fc640 1 -- 172.21.15.49:0/1845949148 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34034e50 con 0x7fcd380ccee0 2024-03-14T21:14:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380ccee0 msgr2=0x7fcd38007ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 --2- 172.21.15.49:0/1845949148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380ccee0 0x7fcd38007ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800d090 tx=0x7fcd340341e0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f412d5fb640 1 -- 172.21.15.49:0/206642954 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4128152c60 msgr2=0x7f41280b8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f412d5fb640 1 --2- 172.21.15.49:0/206642954 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4128152c60 0x7f41280b8160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41100097d0 tx=0x7f411002b960 comp rx=0 tx=0).stop 2024-03-14T21:14:10.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 shutdown_connections 2024-03-14T21:14:10.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 --2- 172.21.15.49:0/1845949148 >> v2:172.21.15.49:3300/0 conn(0x7fcd380cd2c0 0x7fcd380cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 --2- 172.21.15.49:0/1845949148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380ccee0 0x7fcd38007ae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 >> 172.21.15.49:0/1845949148 conn(0x7fcd38009c90 msgr2=0x7fcd3800a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fe5f88b9640 1 Processor -- start 2024-03-14T21:14:10.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f412d5fb640 1 -- 172.21.15.49:0/206642954 shutdown_connections 2024-03-14T21:14:10.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f412d5fb640 1 --2- 172.21.15.49:0/206642954 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4128152c60 0x7f41280b8160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f412d5fb640 1 -- 172.21.15.49:0/206642954 >> 172.21.15.49:0/206642954 conn(0x7f41280bca50 msgr2=0x7f4128069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fe5f88b9640 1 -- start start 2024-03-14T21:14:10.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 shutdown_connections 2024-03-14T21:14:10.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f412d5fb640 1 -- 172.21.15.49:0/206642954 shutdown_connections 2024-03-14T21:14:10.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1bd0ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1b0006eb0 0x7fb1b00d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1bd0ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1b0006eb0 0x7fb1b00d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37732/0 (socket says 172.21.15.49:37732) 2024-03-14T21:14:10.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1bd0ed640 1 -- 172.21.15.49:0/1416618673 learned_addr learned my addr 172.21.15.49:0/1416618673 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1bd8ee640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b00d53d0 0x7fb1b0035150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0173c00 con 0x7fb1b0006eb0 2024-03-14T21:14:10.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b0037c30 con 0x7fb1b0007290 2024-03-14T21:14:10.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b00d0170 con 0x7fb1b00d53d0 2024-03-14T21:14:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f0fa8ff9640 1 -- 172.21.15.49:0/2353116137 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0f98013440 con 0x7f0fa40cc540 2024-03-14T21:14:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f0fa8ff9640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0f9006a8a0 0x7f0f9006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f0fa8ff9640 1 -- 172.21.15.49:0/2353116137 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f980893e0 con 0x7f0fa40cc540 2024-03-14T21:14:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f412d5fb640 1 -- 172.21.15.49:0/206642954 wait complete. 2024-03-14T21:14:10.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7f16db59e640 1 -- 172.21.15.49:0/1254083697 shutdown_connections 2024-03-14T21:14:10.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7f0fabfff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0f9006a8a0 0x7f0f9006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7fb1bd0ed640 1 -- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b00d53d0 msgr2=0x7fb1b0035150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7f0fabfff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0f9006a8a0 0x7f0f9006cd60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000fd0 tx=0x7f0fa0001930 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7f16db59e640 1 -- 172.21.15.49:0/1254083697 wait complete. 2024-03-14T21:14:10.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7fb1bd0ed640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b00d53d0 0x7fb1b0035150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f88b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f406f170 0x7fe5f406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f88b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f40b4820 0x7fe5f40b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7fcd44957640 1 -- 172.21.15.49:0/1845949148 wait complete. 2024-03-14T21:14:10.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f88b9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f41485f0 0x7fe5f41efb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa400d570 con 0x7f0fa40cc540 2024-03-14T21:14:10.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f88b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f406cc30 con 0x7fe5f40b4820 2024-03-14T21:14:10.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.262+0000 7f3658bca640 1 -- 172.21.15.49:0/1363283251 wait complete. 2024-03-14T21:14:10.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.263+0000 7f3658bca640 1 Processor -- start 2024-03-14T21:14:10.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7f3658bca640 1 -- start start 2024-03-14T21:14:10.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f88b9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40b4ee0 con 0x7fe5f406f170 2024-03-14T21:14:10.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3658bca640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c03cd00 0x7f364c187920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3658bca640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c03f870 0x7f364c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3658bca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f364c0418f0 0x7f364c159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3658bca640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c00c5d0 con 0x7f364c03cd00 2024-03-14T21:14:10.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3658bca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c160090 con 0x7f364c0418f0 2024-03-14T21:14:10.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3658bca640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c159680 con 0x7f364c03f870 2024-03-14T21:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f88b9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f41e9f20 con 0x7fe5f41485f0 2024-03-14T21:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f406f170 0x7fe5f406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f406f170 0x7fe5f406c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:33984/0 (socket says 172.21.15.49:33984) 2024-03-14T21:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fcd44957640 1 Processor -- start 2024-03-14T21:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7fb1bd0ed640 1 -- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b0007290 msgr2=0x7fb1b00376f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T21:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7fb1bd0ed640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b0007290 0x7fb1b00376f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7fb1bd0ed640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b806be70 con 0x7fb1b0006eb0 2024-03-14T21:14:10.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.264+0000 7fb1bd0ed640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1b0006eb0 0x7fb1b00d2320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000b60 tx=0x7fb1b8069a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f37fe640 1 -- 172.21.15.49:0/303074979 learned_addr learned my addr 172.21.15.49:0/303074979 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fb1bd8ee640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b00d53d0 0x7fb1b0035150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:10.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c03f870 0x7f364c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c03f870 0x7f364c1584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46672/0 (socket says 172.21.15.49:46672) 2024-03-14T21:14:10.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3652ffd640 1 -- 172.21.15.49:0/1725049585 learned_addr learned my addr 172.21.15.49:0/1725049585 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fb1adffb640 1 -- 172.21.15.49:0/1416618673 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80b6410 con 0x7fb1b0006eb0 2024-03-14T21:14:10.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f3653fff640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f364c0418f0 0x7f364c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fe5f3fff640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f41485f0 0x7fe5f41efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f412d5fb640 1 Processor -- start 2024-03-14T21:14:10.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f412d5fb640 1 -- start start 2024-03-14T21:14:10.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b00d0450 con 0x7fb1b0006eb0 2024-03-14T21:14:10.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7f36537fe640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c03cd00 0x7f364c187920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f16db59e640 1 Processor -- start 2024-03-14T21:14:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fe5f37fe640 1 -- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f41485f0 msgr2=0x7fe5f41efb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fe5f37fe640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f41485f0 0x7fe5f41efb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fe5f37fe640 1 -- 172.21.15.49:0/303074979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f40b4820 msgr2=0x7fe5f40b6ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T21:14:10.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f3653fff640 1 -- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c03f870 msgr2=0x7f364c1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f3653fff640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c03f870 0x7f364c1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f3653fff640 1 -- 172.21.15.49:0/1725049585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c03cd00 msgr2=0x7f364c187920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f3653fff640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c03cd00 0x7f364c187920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f3653fff640 1 -- 172.21.15.49:0/1725049585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3648009400 con 0x7f364c0418f0 2024-03-14T21:14:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f3652ffd640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c03f870 0x7f364c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f36537fe640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c03cd00 0x7f364c187920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.265+0000 7fcd44957640 1 -- start start 2024-03-14T21:14:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b0037f00 con 0x7fb1b0006eb0 2024-03-14T21:14:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fe5f2ffd640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f40b4820 0x7fe5f40b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f3653fff640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f364c0418f0 0x7f364c159140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f365402c880 tx=0x7f3654067690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fe5f37fe640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f40b4820 0x7fe5f40b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fe5f37fe640 1 -- 172.21.15.49:0/303074979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009b30 con 0x7fe5f406f170 2024-03-14T21:14:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f412d5fb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280b3080 0x7f41281e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fb1adffb640 1 -- 172.21.15.49:0/1416618673 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b80b6570 con 0x7fb1b0006eb0 2024-03-14T21:14:10.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7fe5f2ffd640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f40b4820 0x7fe5f40b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:10.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f412d5fb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f412814c520 0x7f412814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f412d5fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4128152c60 0x7f4128151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f4126ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280b3080 0x7f41281e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f4126ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280b3080 0x7f41281e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37760/0 (socket says 172.21.15.49:37760) 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f412d5fb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128151ff0 con 0x7f41280b3080 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f4126ffd640 1 -- 172.21.15.49:0/698560939 learned_addr learned my addr 172.21.15.49:0/698560939 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128141c00 con 0x7f4128152c60 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41281e7700 con 0x7f412814c520 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41267fc640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f412814c520 0x7f412814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41277fe640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4128152c60 0x7f4128151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.266+0000 7f16db59e640 1 -- start start 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16db59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4003b50 0x7f16dc150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16db59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16dc0b60f0 0x7f16dc1e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16db59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16dc0b6570 0x7f16dc1e7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16db59e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc1e4330 con 0x7f16c4003b50 2024-03-14T21:14:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16db59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0b48e0 con 0x7f16dc0b6570 2024-03-14T21:14:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16db59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0b6d30 con 0x7f16dc0b60f0 2024-03-14T21:14:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7fb1adffb640 1 -- 172.21.15.49:0/1416618673 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80b68e0 con 0x7fb1b0006eb0 2024-03-14T21:14:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16da59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4003b50 0x7f16dc150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16da59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4003b50 0x7f16dc150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37768/0 (socket says 172.21.15.49:37768) 2024-03-14T21:14:10.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16da59c640 1 -- 172.21.15.49:0/3574287808 learned_addr learned my addr 172.21.15.49:0/3574287808 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16dad9d640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16dc0b6570 0x7f16dc1e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41267fc640 1 -- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4128152c60 msgr2=0x7f4128151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41267fc640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4128152c60 0x7f4128151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41267fc640 1 -- 172.21.15.49:0/698560939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280b3080 msgr2=0x7f41281e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41267fc640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280b3080 0x7f41281e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41267fc640 1 -- 172.21.15.49:0/698560939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110009400 con 0x7f412814c520 2024-03-14T21:14:10.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16d9d9b640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16dc0b60f0 0x7f16dc1e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f41267fc640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f412814c520 0x7f412814fe40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f412000c770 tx=0x7f412000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7fe5f37fe640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f406f170 0x7fe5f406c6f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b410 tx=0x7fe5e400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16dad9d640 1 -- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16dc0b60f0 msgr2=0x7f16dc1e4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16dad9d640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16dc0b60f0 0x7f16dc1e4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16dad9d640 1 -- 172.21.15.49:0/3574287808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4003b50 msgr2=0x7f16dc150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16dad9d640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4003b50 0x7f16dc150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.267+0000 7f16dad9d640 1 -- 172.21.15.49:0/3574287808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc1e45a0 con 0x7f16dc0b6570 2024-03-14T21:14:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7f411f7fe640 1 -- 172.21.15.49:0/698560939 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120003b70 con 0x7f412814c520 2024-03-14T21:14:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7f16da59c640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4003b50 0x7f16dc150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7f411f7fe640 1 -- 172.21.15.49:0/698560939 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120003cd0 con 0x7f412814c520 2024-03-14T21:14:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41281e79e0 con 0x7f412814c520 2024-03-14T21:14:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b0008160 con 0x7fb1b0006eb0 2024-03-14T21:14:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7f16dad9d640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16dc0b6570 0x7f16dc1e7830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16dc14f710 tx=0x7f16d0007720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fb1adffb640 1 -- 172.21.15.49:0/1416618673 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb1b80b3050 con 0x7fb1b0006eb0 2024-03-14T21:14:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd44957640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd38036310 0x7fcd380343f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd44957640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd380ccee0 0x7fcd3803c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd44957640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380cd2c0 0x7fcd380278b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd44957640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3803aaa0 con 0x7fcd380ccee0 2024-03-14T21:14:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd44957640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38165d70 con 0x7fcd380cd2c0 2024-03-14T21:14:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd44957640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38166050 con 0x7fcd38036310 2024-03-14T21:14:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7f411f7fe640 1 -- 172.21.15.49:0/698560939 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000aa10 con 0x7f412814c520 2024-03-14T21:14:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd38036310 0x7fcd380343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd38036310 0x7fcd380343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:46694/0 (socket says 172.21.15.49:46694) 2024-03-14T21:14:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/995081894 learned_addr learned my addr 172.21.15.49:0/995081894 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.268+0000 7fcd3effd640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd380ccee0 0x7fcd3803c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4128141f30 con 0x7f412814c520 2024-03-14T21:14:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fb1adffb640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb19c06a7f0 0x7fb19c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fb1adffb640 1 -- 172.21.15.49:0/1416618673 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1b81248f0 con 0x7fb1b0006eb0 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd3ffff640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380cd2c0 0x7fcd380278b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380cd2c0 msgr2=0x7fcd380278b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd3f7fe640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380cd2c0 0x7fcd380278b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/995081894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd380ccee0 msgr2=0x7fcd3803c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd3f7fe640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd380ccee0 0x7fcd3803c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd3f7fe640 1 -- 172.21.15.49:0/995081894 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34013a10 con 0x7fcd38036310 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd3f7fe640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd38036310 0x7fcd380343f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd340347f0 tx=0x7fcd340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fb1bc8ec640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb19c06a7f0 0x7fb19c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.270+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4128141d40 con 0x7f412814c520 2024-03-14T21:14:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd2bfff640 1 -- 172.21.15.49:0/995081894 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34046450 con 0x7fcd38036310 2024-03-14T21:14:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.269+0000 7fcd2bfff640 1 -- 172.21.15.49:0/995081894 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd340465b0 con 0x7fcd38036310 2024-03-14T21:14:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.270+0000 7fcd2bfff640 1 -- 172.21.15.49:0/995081894 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34046960 con 0x7fcd38036310 2024-03-14T21:14:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.270+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd38166330 con 0x7fcd38036310 2024-03-14T21:14:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.270+0000 7fb1bc8ec640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb19c06a7f0 0x7fb19c06ccb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1b001f750 tx=0x7fb1a8006b30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.270+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd38035190 con 0x7fcd38036310 2024-03-14T21:14:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.270+0000 7f411f7fe640 1 -- 172.21.15.49:0/698560939 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4120032bc0 con 0x7f412814c520 2024-03-14T21:14:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.270+0000 7f411f7fe640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f410806a7a0 0x7f410806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f411f7fe640 1 -- 172.21.15.49:0/698560939 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4120088300 con 0x7f412814c520 2024-03-14T21:14:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f4126ffd640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f410806a7a0 0x7f410806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd38006860 con 0x7fcd38036310 2024-03-14T21:14:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f363ffff640 1 -- 172.21.15.49:0/1725049585 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540735f0 con 0x7f364c0418f0 2024-03-14T21:14:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f363ffff640 1 -- 172.21.15.49:0/1725049585 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654073750 con 0x7f364c0418f0 2024-03-14T21:14:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f363ffff640 1 -- 172.21.15.49:0/1725049585 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654094460 con 0x7f364c0418f0 2024-03-14T21:14:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f0fa8ff9640 1 -- 172.21.15.49:0/2353116137 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0f98054750 con 0x7f0fa40cc540 2024-03-14T21:14:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c160320 con 0x7f364c0418f0 2024-03-14T21:14:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c157fb0 con 0x7f364c0418f0 2024-03-14T21:14:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7f4126ffd640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f410806a7a0 0x7f410806cc60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4118000f60 tx=0x7f4118007e10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7fcd2bfff640 1 -- 172.21.15.49:0/995081894 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcd34046ac0 con 0x7fcd38036310 2024-03-14T21:14:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7fe5dbfff640 1 -- 172.21.15.49:0/303074979 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4011830 con 0x7fe5f406f170 2024-03-14T21:14:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7fe5dbfff640 1 -- 172.21.15.49:0/303074979 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4011990 con 0x7fe5f406f170 2024-03-14T21:14:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7fe5dbfff640 1 -- 172.21.15.49:0/303074979 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4011d40 con 0x7fe5f406f170 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f4144940 con 0x7fe5f406f170 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f4144ec0 con 0x7fe5f406f170 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f16caffd640 1 -- 172.21.15.49:0/3574287808 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000f830 con 0x7f16dc0b6570 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f16caffd640 1 -- 172.21.15.49:0/3574287808 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d000f990 con 0x7f16dc0b6570 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.271+0000 7f16caffd640 1 -- 172.21.15.49:0/3574287808 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000fd20 con 0x7f16dc0b6570 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc150980 con 0x7f16dc0b6570 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.272+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc0b6a50 con 0x7f16dc0b6570 2024-03-14T21:14:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.273+0000 7fcd2bfff640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd2406a7f0 0x7fcd2406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.273+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c1581a0 con 0x7f364c0418f0 2024-03-14T21:14:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.273+0000 7fcd3effd640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd2406a7f0 0x7fcd2406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.274+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f4073e50 con 0x7fe5f406f170 2024-03-14T21:14:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.274+0000 7fcd3effd640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd2406a7f0 0x7fcd2406ccb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd3815cc20 tx=0x7fcd2c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.274+0000 7f363ffff640 1 -- 172.21.15.49:0/1725049585 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f36540945c0 con 0x7f364c0418f0 2024-03-14T21:14:10.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.274+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16dc1e4730 con 0x7f16dc0b6570 2024-03-14T21:14:10.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7f363ffff640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f363806a8a0 0x7f363806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7f363ffff640 1 -- 172.21.15.49:0/1725049585 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36541022a0 con 0x7f364c0418f0 2024-03-14T21:14:10.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7fe5dbfff640 1 -- 172.21.15.49:0/303074979 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe5e401a400 con 0x7fe5f406f170 2024-03-14T21:14:10.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7f36537fe640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f363806a8a0 0x7f363806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7f16caffd640 1 -- 172.21.15.49:0/3574287808 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f16d001d460 con 0x7f16dc0b6570 2024-03-14T21:14:10.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7fe5dbfff640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe5e006a870 0x7fe5e006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7fe5dbfff640 1 -- 172.21.15.49:0/303074979 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5e4089540 con 0x7fe5f406f170 2024-03-14T21:14:10.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7fe5f2ffd640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe5e006a870 0x7fe5e006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7f16caffd640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f16bc06a7a0 0x7f16bc06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7f36537fe640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f363806a8a0 0x7f363806cd60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3640005880 tx=0x7f3640009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.275+0000 7f16caffd640 1 -- 172.21.15.49:0/3574287808 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16d0088c10 con 0x7f16dc0b6570 2024-03-14T21:14:10.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.276+0000 7f16da59c640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f16bc06a7a0 0x7f16bc06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.276+0000 7fcd2bfff640 1 -- 172.21.15.49:0/995081894 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd34083790 con 0x7fcd38036310 2024-03-14T21:14:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.276+0000 7fe5f2ffd640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe5e006a870 0x7fe5e006cd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40bc2b0 tx=0x7fe5e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.276+0000 7f411f7fe640 1 -- 172.21.15.49:0/698560939 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4128141d40 con 0x7f412814c520 2024-03-14T21:14:10.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.276+0000 7f16da59c640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f16bc06a7a0 0x7f16bc06cc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f16cc006fd0 tx=0x7f16cc031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.277+0000 7fb1adffb640 1 -- 172.21.15.49:0/1416618673 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb1b80f3a30 con 0x7fb1b0006eb0 2024-03-14T21:14:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.279+0000 7fe5dbfff640 1 -- 172.21.15.49:0/303074979 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe5e40548b0 con 0x7fe5f406f170 2024-03-14T21:14:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.281+0000 7fcd2bfff640 1 -- 172.21.15.49:0/995081894 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcd3407f3d0 con 0x7fcd38036310 2024-03-14T21:14:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.283+0000 7f363ffff640 1 -- 172.21.15.49:0/1725049585 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f36540cd730 con 0x7f364c0418f0 2024-03-14T21:14:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.288+0000 7f16caffd640 1 -- 172.21.15.49:0/3574287808 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f16d0053f90 con 0x7f16dc0b6570 2024-03-14T21:14:10.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.647+0000 7fa8c83e7640 1 -- 172.21.15.49:0/3984006867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa8c0066bb0 con 0x7fa8c00b7e10 2024-03-14T21:14:10.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.648+0000 7fa8b6ffd640 1 -- 172.21.15.49:0/3984006867 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8b8038e60 con 0x7fa8c00b7e10 2024-03-14T21:14:10.631 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:10.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.651+0000 7fa8b4ff9640 1 -- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa8a406a920 msgr2=0x7fa8a406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.651+0000 7fa8b4ff9640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa8a406a920 0x7fa8a406cde0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009f2c0 tx=0x7fa8bc009190 comp rx=0 tx=0).stop 2024-03-14T21:14:10.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.651+0000 7fa8b4ff9640 1 -- 172.21.15.49:0/3984006867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b7e10 msgr2=0x7fa8c009ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.651+0000 7fa8b4ff9640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b7e10 0x7fa8c009ce50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8036000 tx=0x7fa8b8004a40 comp rx=0 tx=0).stop 2024-03-14T21:14:10.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.651+0000 7fa8b4ff9640 1 -- 172.21.15.49:0/3984006867 shutdown_connections 2024-03-14T21:14:10.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.651+0000 7fa8b4ff9640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa8a406a920 0x7fa8a406cde0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8c009f2c0 tx=0x7fa8bc009190 comp rx=0 tx=0).stop 2024-03-14T21:14:10.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.651+0000 7fa8b4ff9640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8c0147610 0x7fa8c00a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.652+0000 7fa8b4ff9640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8c00b7e10 0x7fa8c009ce50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.652+0000 7fa8b4ff9640 1 --2- 172.21.15.49:0/3984006867 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8c00b11b0 0x7fa8c01cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.652+0000 7fa8b4ff9640 1 -- 172.21.15.49:0/3984006867 >> 172.21.15.49:0/3984006867 conn(0x7fa8c0069ce0 msgr2=0x7fa8c009ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.652+0000 7fa8b4ff9640 1 -- 172.21.15.49:0/3984006867 shutdown_connections 2024-03-14T21:14:10.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.653+0000 7fa8b4ff9640 1 -- 172.21.15.49:0/3984006867 wait complete. 2024-03-14T21:14:10.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-03-14T21:14:10.655 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.690+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f16dc073e50 con 0x7f16dc0b6570 2024-03-14T21:14:10.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.691+0000 7f16caffd640 1 -- 172.21.15.49:0/3574287808 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16d00575b0 con 0x7f16dc0b6570 2024-03-14T21:14:10.674 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:10.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcd38028c00 con 0x7fcd38036310 2024-03-14T21:14:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f16bc06a7a0 msgr2=0x7f16bc06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f16bc06a7a0 0x7f16bc06cc60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f16cc006fd0 tx=0x7f16cc031000 comp rx=0 tx=0).stop 2024-03-14T21:14:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16dc0b6570 msgr2=0x7f16dc1e7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16dc0b6570 0x7f16dc1e7830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16dc14f710 tx=0x7f16d0007720 comp rx=0 tx=0).stop 2024-03-14T21:14:10.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 shutdown_connections 2024-03-14T21:14:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f16bc06a7a0 0x7f16bc06cc60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16dc0b6570 0x7f16dc1e7830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16dc0b60f0 0x7f16dc1e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 --2- 172.21.15.49:0/3574287808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f16c4003b50 0x7f16dc150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 >> 172.21.15.49:0/3574287808 conn(0x7f16dc0b9a90 msgr2=0x7f16dc146630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7fcd2bfff640 1 -- 172.21.15.49:0/995081894 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd3404b090 con 0x7fcd38036310 2024-03-14T21:14:10.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.693+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0fa403f560 con 0x7f0fa40cc540 2024-03-14T21:14:10.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.694+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 shutdown_connections 2024-03-14T21:14:10.679 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:10.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.694+0000 7f16db59e640 1 -- 172.21.15.49:0/3574287808 wait complete. 2024-03-14T21:14:10.680 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:10.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.694+0000 7f0fa8ff9640 1 -- 172.21.15.49:0/2353116137 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f98057d70 con 0x7f0fa40cc540 2024-03-14T21:14:10.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd2406a7f0 msgr2=0x7fcd2406ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd2406a7f0 0x7fcd2406ccb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd3815cc20 tx=0x7fcd2c009190 comp rx=0 tx=0).stop 2024-03-14T21:14:10.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd38036310 msgr2=0x7fcd380343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd38036310 0x7fcd380343f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd340347f0 tx=0x7fcd340046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0f9006a8a0 msgr2=0x7f0f9006cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 shutdown_connections 2024-03-14T21:14:10.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0f9006a8a0 0x7f0f9006cd60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000fd0 tx=0x7f0fa0001930 comp rx=0 tx=0).stop 2024-03-14T21:14:10.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd2406a7f0 0x7fcd2406ccb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd380cd2c0 0x7fcd380278b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd380ccee0 0x7fcd3803c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 --2- 172.21.15.49:0/995081894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd38036310 0x7fcd380343f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 >> 172.21.15.49:0/995081894 conn(0x7fcd38009c90 msgr2=0x7fcd380c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa40cc540 msgr2=0x7f0fa416e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa40cc540 0x7f0fa416e8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40d2490 tx=0x7f0f9800fbb0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 shutdown_connections 2024-03-14T21:14:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f0f9006a8a0 0x7f0f9006cd60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fa416ee20 0x7f0fa40d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fa40cc540 0x7f0fa416e8e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 --2- 172.21.15.49:0/2353116137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f9c009970 0x7f0fa40cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 shutdown_connections 2024-03-14T21:14:10.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7fcd44957640 1 -- 172.21.15.49:0/995081894 wait complete. 2024-03-14T21:14:10.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 >> 172.21.15.49:0/2353116137 conn(0x7f0fa400a550 msgr2=0x7f0fa4022fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.697+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 shutdown_connections 2024-03-14T21:14:10.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.698+0000 7f0fb15ff640 1 -- 172.21.15.49:0/2353116137 wait complete. 2024-03-14T21:14:10.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.698+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb1b0173d40 con 0x7fb1b0006eb0 2024-03-14T21:14:10.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.699+0000 7fb1adffb640 1 -- 172.21.15.49:0/1416618673 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1b0173d40 con 0x7fb1b0006eb0 2024-03-14T21:14:10.686 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.701+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe5f4145ff0 con 0x7fe5f406f170 2024-03-14T21:14:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.701+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f41281e7f00 con 0x7f412814c520 2024-03-14T21:14:10.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.701+0000 7fe5dbfff640 1 -- 172.21.15.49:0/303074979 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5e4015090 con 0x7fe5f406f170 2024-03-14T21:14:10.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.702+0000 7f411f7fe640 1 -- 172.21.15.49:0/698560939 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f41281e7f00 con 0x7f412814c520 2024-03-14T21:14:10.688 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:10.688 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.703+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb19c06a7f0 msgr2=0x7fb19c06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.703+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb19c06a7f0 0x7fb19c06ccb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1b001f750 tx=0x7fb1a8006b30 comp rx=0 tx=0).stop 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.703+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1b0006eb0 msgr2=0x7fb1b00d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.703+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1b0006eb0 0x7fb1b00d2320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8000b60 tx=0x7fb1b8069a00 comp rx=0 tx=0).stop 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 shutdown_connections 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb19c06a7f0 0x7fb19c06ccb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1b00d53d0 0x7fb1b0035150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1b0007290 0x7fb1b00376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 --2- 172.21.15.49:0/1416618673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1b0006eb0 0x7fb1b00d2320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 >> 172.21.15.49:0/1416618673 conn(0x7fb1b003ede0 msgr2=0x7fb1b00d3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 shutdown_connections 2024-03-14T21:14:10.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.704+0000 7fb1be0ef640 1 -- 172.21.15.49:0/1416618673 wait complete. 2024-03-14T21:14:10.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f410806a7a0 msgr2=0x7f410806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f410806a7a0 0x7f410806cc60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4118000f60 tx=0x7f4118007e10 comp rx=0 tx=0).stop 2024-03-14T21:14:10.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f364c0056e0 con 0x7f364c0418f0 2024-03-14T21:14:10.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f412814c520 msgr2=0x7f412814fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f412814c520 0x7f412814fe40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f412000c770 tx=0x7f412000cc30 comp rx=0 tx=0).stop 2024-03-14T21:14:10.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 shutdown_connections 2024-03-14T21:14:10.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f410806a7a0 0x7f410806cc60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4128152c60 0x7f4128151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f412814c520 0x7f412814fe40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 --2- 172.21.15.49:0/698560939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280b3080 0x7f41281e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe5e006a870 msgr2=0x7fe5e006cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe5e006a870 0x7fe5e006cd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40bc2b0 tx=0x7fe5e8008040 comp rx=0 tx=0).stop 2024-03-14T21:14:10.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 >> 172.21.15.49:0/698560939 conn(0x7f41280bca50 msgr2=0x7f41280baa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f406f170 msgr2=0x7fe5f406c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.705+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f406f170 0x7fe5f406c6f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b410 tx=0x7fe5e400b8d0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 shutdown_connections 2024-03-14T21:14:10.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7f412d5fb640 1 -- 172.21.15.49:0/698560939 wait complete. 2024-03-14T21:14:10.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7f363ffff640 1 -- 172.21.15.49:0/1725049585 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f36540d0d50 con 0x7f364c0418f0 2024-03-14T21:14:10.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 shutdown_connections 2024-03-14T21:14:10.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe5e006a870 0x7fe5e006cd30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5f41485f0 0x7fe5f41efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f40b4820 0x7fe5f40b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 --2- 172.21.15.49:0/303074979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f406f170 0x7fe5f406c6f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.695 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-03-14T21:14:10.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 >> 172.21.15.49:0/303074979 conn(0x7fe5f4070f20 msgr2=0x7fe5f40bc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 shutdown_connections 2024-03-14T21:14:10.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.706+0000 7fe5f88b9640 1 -- 172.21.15.49:0/303074979 wait complete. 2024-03-14T21:14:10.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.708+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f363806a8a0 msgr2=0x7f363806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.708+0000 7f3658bca640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f363806a8a0 0x7f363806cd60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3640005880 tx=0x7f3640009040 comp rx=0 tx=0).stop 2024-03-14T21:14:10.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.708+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f364c0418f0 msgr2=0x7f364c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.708+0000 7f3658bca640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f364c0418f0 0x7f364c159140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f365402c880 tx=0x7f3654067690 comp rx=0 tx=0).stop 2024-03-14T21:14:10.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 shutdown_connections 2024-03-14T21:14:10.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f363806a8a0 0x7f363806cd60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f364c0418f0 0x7f364c159140 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f364c03f870 0x7f364c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 --2- 172.21.15.49:0/1725049585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364c03cd00 0x7f364c187920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 >> 172.21.15.49:0/1725049585 conn(0x7f364c046970 msgr2=0x7f364c15cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 shutdown_connections 2024-03-14T21:14:10.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.709+0000 7f3658bca640 1 -- 172.21.15.49:0/1725049585 wait complete. 2024-03-14T21:14:10.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-03-14T21:14:10.702 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-03-14T21:14:10.703 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-03-14T21:14:10.703 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-03-14T21:14:10.707 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-03-14T21:14:10.707 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-03-14T21:14:10.710 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-03-14T21:14:10.711 DEBUG:teuthology.parallel:result is None 2024-03-14T21:14:10.711 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-14T21:14:10.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T21:14:10.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.873+0000 7f30d32e9640 1 Processor -- start 2024-03-14T21:14:10.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d32e9640 1 -- start start 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d32e9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30cc0baef0 0x7f30cc1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d32e9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30cc0bc940 0x7f30cc155710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d32e9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30cc06a6e0 con 0x7f30cc06b620 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d32e9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30cc06ad10 con 0x7f30cc0baef0 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d32e9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30cc0bc2b0 con 0x7f30cc0bc940 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d1085640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30cc0baef0 0x7f30cc1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.874+0000 7f30d1085640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30cc0baef0 0x7f30cc1480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43774/0 (socket says 172.21.15.49:43774) 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.875+0000 7f30d1085640 1 -- 172.21.15.49:0/2372649148 learned_addr learned my addr 172.21.15.49:0/2372649148 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.875+0000 7f30d1886640 1 --2- 172.21.15.49:0/2372649148 >> v2:172.21.15.97:3300/0 conn(0x7f30cc0bc940 0x7f30cc155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.875+0000 7f30d1085640 1 -- 172.21.15.49:0/2372649148 >> v2:172.21.15.97:3300/0 conn(0x7f30cc0bc940 msgr2=0x7f30cc155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.875+0000 7f30d1085640 1 --2- 172.21.15.49:0/2372649148 >> v2:172.21.15.97:3300/0 conn(0x7f30cc0bc940 0x7f30cc155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.875+0000 7f30d1085640 1 -- 172.21.15.49:0/2372649148 >> v1:172.21.15.97:6789/0 conn(0x7f30cc06b620 legacy=0x7f30cc0bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.875+0000 7f30d1085640 1 -- 172.21.15.49:0/2372649148 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc06bab0 con 0x7f30cc0baef0 2024-03-14T21:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.875+0000 7f30d1085640 1 --2- 172.21.15.49:0/2372649148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1480b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f30cc14f480 tx=0x7f30c002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e152517641915fc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30bffff640 1 -- 172.21.15.49:0/2372649148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00317e0 con 0x7f30cc0baef0 2024-03-14T21:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30bffff640 1 -- 172.21.15.49:0/2372649148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0031940 con 0x7f30cc0baef0 2024-03-14T21:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30bffff640 1 -- 172.21.15.49:0/2372649148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0031cd0 con 0x7f30cc0baef0 2024-03-14T21:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 -- 172.21.15.49:0/2372649148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 msgr2=0x7f30cc1480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 --2- 172.21.15.49:0/2372649148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1480b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f30cc14f480 tx=0x7f30c002b880 comp rx=0 tx=0).stop 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 -- 172.21.15.49:0/2372649148 shutdown_connections 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 --2- 172.21.15.49:0/2372649148 >> v2:172.21.15.97:3300/0 conn(0x7f30cc0bc940 0x7f30cc155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 --2- 172.21.15.49:0/2372649148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1480b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 -- 172.21.15.49:0/2372649148 >> 172.21.15.49:0/2372649148 conn(0x7f30cc0bcd80 msgr2=0x7f30cc0bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 -- 172.21.15.49:0/2372649148 shutdown_connections 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.876+0000 7f30d32e9640 1 -- 172.21.15.49:0/2372649148 wait complete. 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.877+0000 7f30d32e9640 1 Processor -- start 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.877+0000 7f30d32e9640 1 -- start start 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d32e9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d32e9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30cc0bc940 0x7f30cc1dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d32e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30cc0b0400 0x7f30cc0a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d32e9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc0b0920 con 0x7f30cc0baef0 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d32e9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc14c7b0 con 0x7f30cc0b0400 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d32e9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc1e3b80 con 0x7f30cc0bc940 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1e09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43786/0 (socket says 172.21.15.49:43786) 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 -- 172.21.15.49:0/803712628 learned_addr learned my addr 172.21.15.49:0/803712628 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1886640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30cc0b0400 0x7f30cc0a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d0884640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30cc0bc940 0x7f30cc1dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 -- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30cc0bc940 msgr2=0x7f30cc1dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30cc0bc940 0x7f30cc1dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 -- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30cc0b0400 msgr2=0x7f30cc0a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30cc0b0400 0x7f30cc0a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.878+0000 7f30d1085640 1 -- 172.21.15.49:0/803712628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c00094d0 con 0x7f30cc0baef0 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.879+0000 7f30d1085640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1e09a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.879+0000 7f30bdffb640 1 -- 172.21.15.49:0/803712628 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0032900 con 0x7f30cc0baef0 2024-03-14T21:14:10.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.879+0000 7f30bdffb640 1 -- 172.21.15.49:0/803712628 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0032a60 con 0x7f30cc0baef0 2024-03-14T21:14:10.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.879+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30cc1e0300 con 0x7f30cc0baef0 2024-03-14T21:14:10.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.879+0000 7f30bdffb640 1 -- 172.21.15.49:0/803712628 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0032e10 con 0x7f30cc0baef0 2024-03-14T21:14:10.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.879+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30cc1e3810 con 0x7f30cc0baef0 2024-03-14T21:14:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.880+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30cc1e3390 con 0x7f30cc0baef0 2024-03-14T21:14:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.880+0000 7f30bdffb640 1 -- 172.21.15.49:0/803712628 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f30c00453f0 con 0x7f30cc0baef0 2024-03-14T21:14:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.881+0000 7f30bdffb640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f30b006a7d0 0x7f30b006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.881+0000 7f30bdffb640 1 -- 172.21.15.49:0/803712628 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30c00ab180 con 0x7f30cc0baef0 2024-03-14T21:14:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.881+0000 7f30d0884640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f30b006a7d0 0x7f30b006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.881+0000 7f30d0884640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f30b006a7d0 0x7f30b006cc90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30cc149bf0 tx=0x7f30c8007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:10.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:10.885+0000 7f30bdffb640 1 -- 172.21.15.49:0/803712628 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f30cc1e3390 con 0x7f30cc0baef0 2024-03-14T21:14:11.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.064+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30cc067680 con 0x7f30b006a7d0 2024-03-14T21:14:11.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.071+0000 7f30bdffb640 1 -- 172.21.15.49:0/803712628 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264577 (secure 0 0 0) 0x7f30cc067680 con 0x7f30b006a7d0 2024-03-14T21:14:11.057 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:11.059 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T21:14:11.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f30b006a7d0 msgr2=0x7f30b006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f30b006a7d0 0x7f30b006cc90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30cc149bf0 tx=0x7f30c8007e30 comp rx=0 tx=0).stop 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 msgr2=0x7f30cc1e09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1e09a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c00046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 shutdown_connections 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f30b006a7d0 0x7f30b006cc90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30cc0b0400 0x7f30cc0a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30cc0bc940 0x7f30cc1dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 --2- 172.21.15.49:0/803712628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30cc0baef0 0x7f30cc1e09a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 >> 172.21.15.49:0/803712628 conn(0x7f30cc0bcd80 msgr2=0x7f30cc1472f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 shutdown_connections 2024-03-14T21:14:11.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.074+0000 7f30d32e9640 1 -- 172.21.15.49:0/803712628 wait complete. 2024-03-14T21:14:11.068 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-03-14T21:14:10.911083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235460,"kb_used_data":2916,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754739260,"statfs":{"total":773094113280,"available":772853002240,"internally_reserved":0,"allocated":2985984,"data_stored":1789393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002341"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954700+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T21:14:04.954700+0000","last_peered":"2024-03-14T21:14:04.954700+0000","last_clean":"2024-03-14T21:14:04.954700+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T21:14:04.954700+0000","last_undegraded":"2024-03-14T21:14:04.954700+0000","last_fullsized":"2024-03-14T21:14:04.954700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:57:09.325211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017662+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T21:14:04.017662+0000","last_peered":"2024-03-14T21:14:04.017662+0000","last_clean":"2024-03-14T21:14:04.017662+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T21:14:04.017662+0000","last_undegraded":"2024-03-14T21:14:04.017662+0000","last_fullsized":"2024-03-14T21:14:04.017662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928709+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T21:14:04.928709+0000","last_peered":"2024-03-14T21:14:04.928709+0000","last_clean":"2024-03-14T21:14:04.928709+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T21:14:04.928709+0000","last_undegraded":"2024-03-14T21:14:04.928709+0000","last_fullsized":"2024-03-14T21:14:04.928709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:08:55.492956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019894+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T21:14:04.019894+0000","last_peered":"2024-03-14T21:14:04.019894+0000","last_clean":"2024-03-14T21:14:04.019894+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T21:14:04.019894+0000","last_undegraded":"2024-03-14T21:14:04.019894+0000","last_fullsized":"2024-03-14T21:14:04.019894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954528+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T21:14:04.954528+0000","last_peered":"2024-03-14T21:14:04.954528+0000","last_clean":"2024-03-14T21:14:04.954528+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T21:14:04.954528+0000","last_undegraded":"2024-03-14T21:14:04.954528+0000","last_fullsized":"2024-03-14T21:14:04.954528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019354+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T21:14:04.019354+0000","last_peered":"2024-03-14T21:14:04.019354+0000","last_clean":"2024-03-14T21:14:04.019354+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T21:14:04.019354+0000","last_undegraded":"2024-03-14T21:14:04.019354+0000","last_fullsized":"2024-03-14T21:14:04.019354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954494+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T21:14:04.954494+0000","last_peered":"2024-03-14T21:14:04.954494+0000","last_clean":"2024-03-14T21:14:04.954494+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T21:14:04.954494+0000","last_undegraded":"2024-03-14T21:14:04.954494+0000","last_fullsized":"2024-03-14T21:14:04.954494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243611+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T21:14:04.243611+0000","last_peered":"2024-03-14T21:14:04.243611+0000","last_clean":"2024-03-14T21:14:04.243611+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T21:14:04.243611+0000","last_undegraded":"2024-03-14T21:14:04.243611+0000","last_fullsized":"2024-03-14T21:14:04.243611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954715+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T21:14:04.954715+0000","last_peered":"2024-03-14T21:14:04.954715+0000","last_clean":"2024-03-14T21:14:04.954715+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T21:14:04.954715+0000","last_undegraded":"2024-03-14T21:14:04.954715+0000","last_fullsized":"2024-03-14T21:14:04.954715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928838+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T21:14:04.928838+0000","last_peered":"2024-03-14T21:14:04.928838+0000","last_clean":"2024-03-14T21:14:04.928838+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T21:14:04.928838+0000","last_undegraded":"2024-03-14T21:14:04.928838+0000","last_fullsized":"2024-03-14T21:14:04.928838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928822+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T21:14:04.928822+0000","last_peered":"2024-03-14T21:14:04.928822+0000","last_clean":"2024-03-14T21:14:04.928822+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T21:14:04.928822+0000","last_undegraded":"2024-03-14T21:14:04.928822+0000","last_fullsized":"2024-03-14T21:14:04.928822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019337+0000","last_change":"2024-03-14T21:14:03.034043+0000","last_active":"2024-03-14T21:14:04.019337+0000","last_peered":"2024-03-14T21:14:04.019337+0000","last_clean":"2024-03-14T21:14:04.019337+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T21:14:04.019337+0000","last_undegraded":"2024-03-14T21:14:04.019337+0000","last_fullsized":"2024-03-14T21:14:04.019337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:33.787920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243838+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T21:14:04.243838+0000","last_peered":"2024-03-14T21:14:04.243838+0000","last_clean":"2024-03-14T21:14:04.243838+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T21:14:04.243838+0000","last_undegraded":"2024-03-14T21:14:04.243838+0000","last_fullsized":"2024-03-14T21:14:04.243838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928654+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T21:14:04.928654+0000","last_peered":"2024-03-14T21:14:04.928654+0000","last_clean":"2024-03-14T21:14:04.928654+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T21:14:04.928654+0000","last_undegraded":"2024-03-14T21:14:04.928654+0000","last_fullsized":"2024-03-14T21:14:04.928654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017669+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T21:14:04.017669+0000","last_peered":"2024-03-14T21:14:04.017669+0000","last_clean":"2024-03-14T21:14:04.017669+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T21:14:04.017669+0000","last_undegraded":"2024-03-14T21:14:04.017669+0000","last_fullsized":"2024-03-14T21:14:04.017669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018774+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T21:14:04.018774+0000","last_peered":"2024-03-14T21:14:04.018774+0000","last_clean":"2024-03-14T21:14:04.018774+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T21:14:04.018774+0000","last_undegraded":"2024-03-14T21:14:04.018774+0000","last_fullsized":"2024-03-14T21:14:04.018774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019994+0000","last_change":"2024-03-14T21:14:03.035328+0000","last_active":"2024-03-14T21:14:04.019994+0000","last_peered":"2024-03-14T21:14:04.019994+0000","last_clean":"2024-03-14T21:14:04.019994+0000","last_became_active":"2024-03-14T21:14:03.034934+0000","last_became_peered":"2024-03-14T21:14:03.034934+0000","last_unstale":"2024-03-14T21:14:04.019994+0000","last_undegraded":"2024-03-14T21:14:04.019994+0000","last_fullsized":"2024-03-14T21:14:04.019994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930124+0000","last_change":"2024-03-14T21:14:03.029823+0000","last_active":"2024-03-14T21:14:04.930124+0000","last_peered":"2024-03-14T21:14:04.930124+0000","last_clean":"2024-03-14T21:14:04.930124+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T21:14:04.930124+0000","last_undegraded":"2024-03-14T21:14:04.930124+0000","last_fullsized":"2024-03-14T21:14:04.930124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:47.131679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954641+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T21:14:04.954641+0000","last_peered":"2024-03-14T21:14:04.954641+0000","last_clean":"2024-03-14T21:14:04.954641+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T21:14:04.954641+0000","last_undegraded":"2024-03-14T21:14:04.954641+0000","last_fullsized":"2024-03-14T21:14:04.954641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928783+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T21:14:04.928783+0000","last_peered":"2024-03-14T21:14:04.928783+0000","last_clean":"2024-03-14T21:14:04.928783+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T21:14:04.928783+0000","last_undegraded":"2024-03-14T21:14:04.928783+0000","last_fullsized":"2024-03-14T21:14:04.928783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928673+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T21:14:04.928673+0000","last_peered":"2024-03-14T21:14:04.928673+0000","last_clean":"2024-03-14T21:14:04.928673+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T21:14:04.928673+0000","last_undegraded":"2024-03-14T21:14:04.928673+0000","last_fullsized":"2024-03-14T21:14:04.928673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019386+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T21:14:04.019386+0000","last_peered":"2024-03-14T21:14:04.019386+0000","last_clean":"2024-03-14T21:14:04.019386+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T21:14:04.019386+0000","last_undegraded":"2024-03-14T21:14:04.019386+0000","last_fullsized":"2024-03-14T21:14:04.019386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243662+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T21:14:04.243662+0000","last_peered":"2024-03-14T21:14:04.243662+0000","last_clean":"2024-03-14T21:14:04.243662+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T21:14:04.243662+0000","last_undegraded":"2024-03-14T21:14:04.243662+0000","last_fullsized":"2024-03-14T21:14:04.243662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018867+0000","last_change":"2024-03-14T21:14:03.038694+0000","last_active":"2024-03-14T21:14:04.018867+0000","last_peered":"2024-03-14T21:14:04.018867+0000","last_clean":"2024-03-14T21:14:04.018867+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T21:14:04.018867+0000","last_undegraded":"2024-03-14T21:14:04.018867+0000","last_fullsized":"2024-03-14T21:14:04.018867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243990+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T21:14:04.243990+0000","last_peered":"2024-03-14T21:14:04.243990+0000","last_clean":"2024-03-14T21:14:04.243990+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T21:14:04.243990+0000","last_undegraded":"2024-03-14T21:14:04.243990+0000","last_fullsized":"2024-03-14T21:14:04.243990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928859+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T21:14:04.928859+0000","last_peered":"2024-03-14T21:14:04.928859+0000","last_clean":"2024-03-14T21:14:04.928859+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T21:14:04.928859+0000","last_undegraded":"2024-03-14T21:14:04.928859+0000","last_fullsized":"2024-03-14T21:14:04.928859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017498+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T21:14:04.017498+0000","last_peered":"2024-03-14T21:14:04.017498+0000","last_clean":"2024-03-14T21:14:04.017498+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T21:14:04.017498+0000","last_undegraded":"2024-03-14T21:14:04.017498+0000","last_fullsized":"2024-03-14T21:14:04.017498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243915+0000","last_change":"2024-03-14T21:14:03.040690+0000","last_active":"2024-03-14T21:14:04.243915+0000","last_peered":"2024-03-14T21:14:04.243915+0000","last_clean":"2024-03-14T21:14:04.243915+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T21:14:04.243915+0000","last_undegraded":"2024-03-14T21:14:04.243915+0000","last_fullsized":"2024-03-14T21:14:04.243915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928682+0000","last_change":"2024-03-14T21:14:03.038048+0000","last_active":"2024-03-14T21:14:04.928682+0000","last_peered":"2024-03-14T21:14:04.928682+0000","last_clean":"2024-03-14T21:14:04.928682+0000","last_became_active":"2024-03-14T21:14:03.037273+0000","last_became_peered":"2024-03-14T21:14:03.037273+0000","last_unstale":"2024-03-14T21:14:04.928682+0000","last_undegraded":"2024-03-14T21:14:04.928682+0000","last_fullsized":"2024-03-14T21:14:04.928682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019469+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T21:14:04.019469+0000","last_peered":"2024-03-14T21:14:04.019469+0000","last_clean":"2024-03-14T21:14:04.019469+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T21:14:04.019469+0000","last_undegraded":"2024-03-14T21:14:04.019469+0000","last_fullsized":"2024-03-14T21:14:04.019469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930068+0000","last_change":"2024-03-14T21:14:03.029847+0000","last_active":"2024-03-14T21:14:04.930068+0000","last_peered":"2024-03-14T21:14:04.930068+0000","last_clean":"2024-03-14T21:14:04.930068+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T21:14:04.930068+0000","last_undegraded":"2024-03-14T21:14:04.930068+0000","last_fullsized":"2024-03-14T21:14:04.930068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928688+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T21:14:04.928688+0000","last_peered":"2024-03-14T21:14:04.928688+0000","last_clean":"2024-03-14T21:14:04.928688+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T21:14:04.928688+0000","last_undegraded":"2024-03-14T21:14:04.928688+0000","last_fullsized":"2024-03-14T21:14:04.928688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018978+0000","last_change":"2024-03-14T21:14:03.037915+0000","last_active":"2024-03-14T21:14:04.018978+0000","last_peered":"2024-03-14T21:14:04.018978+0000","last_clean":"2024-03-14T21:14:04.018978+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T21:14:04.018978+0000","last_undegraded":"2024-03-14T21:14:04.018978+0000","last_fullsized":"2024-03-14T21:14:04.018978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243993+0000","last_change":"2024-03-14T21:14:02.655762+0000","last_active":"2024-03-14T21:14:04.243993+0000","last_peered":"2024-03-14T21:14:04.243993+0000","last_clean":"2024-03-14T21:14:04.243993+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T21:14:04.243993+0000","last_undegraded":"2024-03-14T21:14:04.243993+0000","last_fullsized":"2024-03-14T21:14:04.243993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928690+0000","last_change":"2024-03-14T21:14:03.033091+0000","last_active":"2024-03-14T21:14:04.928690+0000","last_peered":"2024-03-14T21:14:04.928690+0000","last_clean":"2024-03-14T21:14:04.928690+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T21:14:04.928690+0000","last_undegraded":"2024-03-14T21:14:04.928690+0000","last_fullsized":"2024-03-14T21:14:04.928690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928765+0000","last_change":"2024-03-14T21:14:02.662899+0000","last_active":"2024-03-14T21:14:04.928765+0000","last_peered":"2024-03-14T21:14:04.928765+0000","last_clean":"2024-03-14T21:14:04.928765+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T21:14:04.928765+0000","last_undegraded":"2024-03-14T21:14:04.928765+0000","last_fullsized":"2024-03-14T21:14:04.928765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243862+0000","last_change":"2024-03-14T21:14:02.660574+0000","last_active":"2024-03-14T21:14:04.243862+0000","last_peered":"2024-03-14T21:14:04.243862+0000","last_clean":"2024-03-14T21:14:04.243862+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T21:14:04.243862+0000","last_undegraded":"2024-03-14T21:14:04.243862+0000","last_fullsized":"2024-03-14T21:14:04.243862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243590+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T21:14:04.243590+0000","last_peered":"2024-03-14T21:14:04.243590+0000","last_clean":"2024-03-14T21:14:04.243590+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T21:14:04.243590+0000","last_undegraded":"2024-03-14T21:14:04.243590+0000","last_fullsized":"2024-03-14T21:14:04.243590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018961+0000","last_change":"2024-03-14T21:14:02.661303+0000","last_active":"2024-03-14T21:14:04.018961+0000","last_peered":"2024-03-14T21:14:04.018961+0000","last_clean":"2024-03-14T21:14:04.018961+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T21:14:04.018961+0000","last_undegraded":"2024-03-14T21:14:04.018961+0000","last_fullsized":"2024-03-14T21:14:04.018961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017394+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T21:14:04.017394+0000","last_peered":"2024-03-14T21:14:04.017394+0000","last_clean":"2024-03-14T21:14:04.017394+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T21:14:04.017394+0000","last_undegraded":"2024-03-14T21:14:04.017394+0000","last_fullsized":"2024-03-14T21:14:04.017394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928875+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T21:14:04.928875+0000","last_peered":"2024-03-14T21:14:04.928875+0000","last_clean":"2024-03-14T21:14:04.928875+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T21:14:04.928875+0000","last_undegraded":"2024-03-14T21:14:04.928875+0000","last_fullsized":"2024-03-14T21:14:04.928875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019922+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T21:14:04.019922+0000","last_peered":"2024-03-14T21:14:04.019922+0000","last_clean":"2024-03-14T21:14:04.019922+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T21:14:04.019922+0000","last_undegraded":"2024-03-14T21:14:04.019922+0000","last_fullsized":"2024-03-14T21:14:04.019922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019816+0000","last_change":"2024-03-14T21:14:02.660384+0000","last_active":"2024-03-14T21:14:04.019816+0000","last_peered":"2024-03-14T21:14:04.019816+0000","last_clean":"2024-03-14T21:14:04.019816+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T21:14:04.019816+0000","last_undegraded":"2024-03-14T21:14:04.019816+0000","last_fullsized":"2024-03-14T21:14:04.019816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928734+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T21:14:04.928734+0000","last_peered":"2024-03-14T21:14:04.928734+0000","last_clean":"2024-03-14T21:14:04.928734+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T21:14:04.928734+0000","last_undegraded":"2024-03-14T21:14:04.928734+0000","last_fullsized":"2024-03-14T21:14:04.928734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:57:33.155885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017568+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T21:14:04.017568+0000","last_peered":"2024-03-14T21:14:04.017568+0000","last_clean":"2024-03-14T21:14:04.017568+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T21:14:04.017568+0000","last_undegraded":"2024-03-14T21:14:04.017568+0000","last_fullsized":"2024-03-14T21:14:04.017568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019326+0000","last_change":"2024-03-14T21:14:03.032533+0000","last_active":"2024-03-14T21:14:04.019326+0000","last_peered":"2024-03-14T21:14:04.019326+0000","last_clean":"2024-03-14T21:14:04.019326+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T21:14:04.019326+0000","last_undegraded":"2024-03-14T21:14:04.019326+0000","last_fullsized":"2024-03-14T21:14:04.019326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:41.581386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928805+0000","last_change":"2024-03-14T21:14:02.659028+0000","last_active":"2024-03-14T21:14:04.928805+0000","last_peered":"2024-03-14T21:14:04.928805+0000","last_clean":"2024-03-14T21:14:04.928805+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T21:14:04.928805+0000","last_undegraded":"2024-03-14T21:14:04.928805+0000","last_fullsized":"2024-03-14T21:14:04.928805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954595+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T21:14:04.954595+0000","last_peered":"2024-03-14T21:14:04.954595+0000","last_clean":"2024-03-14T21:14:04.954595+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T21:14:04.954595+0000","last_undegraded":"2024-03-14T21:14:04.954595+0000","last_fullsized":"2024-03-14T21:14:04.954595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928699+0000","last_change":"2024-03-14T21:14:02.663139+0000","last_active":"2024-03-14T21:14:04.928699+0000","last_peered":"2024-03-14T21:14:04.928699+0000","last_clean":"2024-03-14T21:14:04.928699+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T21:14:04.928699+0000","last_undegraded":"2024-03-14T21:14:04.928699+0000","last_fullsized":"2024-03-14T21:14:04.928699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:25:31.095452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928664+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T21:14:04.928664+0000","last_peered":"2024-03-14T21:14:04.928664+0000","last_clean":"2024-03-14T21:14:04.928664+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T21:14:04.928664+0000","last_undegraded":"2024-03-14T21:14:04.928664+0000","last_fullsized":"2024-03-14T21:14:04.928664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019604+0000","last_change":"2024-03-14T21:14:02.659773+0000","last_active":"2024-03-14T21:14:04.019604+0000","last_peered":"2024-03-14T21:14:04.019604+0000","last_clean":"2024-03-14T21:14:04.019604+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T21:14:04.019604+0000","last_undegraded":"2024-03-14T21:14:04.019604+0000","last_fullsized":"2024-03-14T21:14:04.019604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954543+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T21:14:04.954543+0000","last_peered":"2024-03-14T21:14:04.954543+0000","last_clean":"2024-03-14T21:14:04.954543+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T21:14:04.954543+0000","last_undegraded":"2024-03-14T21:14:04.954543+0000","last_fullsized":"2024-03-14T21:14:04.954543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928731+0000","last_change":"2024-03-14T21:14:02.663585+0000","last_active":"2024-03-14T21:14:04.928731+0000","last_peered":"2024-03-14T21:14:04.928731+0000","last_clean":"2024-03-14T21:14:04.928731+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T21:14:04.928731+0000","last_undegraded":"2024-03-14T21:14:04.928731+0000","last_fullsized":"2024-03-14T21:14:04.928731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928713+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T21:14:04.928713+0000","last_peered":"2024-03-14T21:14:04.928713+0000","last_clean":"2024-03-14T21:14:04.928713+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T21:14:04.928713+0000","last_undegraded":"2024-03-14T21:14:04.928713+0000","last_fullsized":"2024-03-14T21:14:04.928713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930043+0000","last_change":"2024-03-14T21:14:02.654931+0000","last_active":"2024-03-14T21:14:04.930043+0000","last_peered":"2024-03-14T21:14:04.930043+0000","last_clean":"2024-03-14T21:14:04.930043+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T21:14:04.930043+0000","last_undegraded":"2024-03-14T21:14:04.930043+0000","last_fullsized":"2024-03-14T21:14:04.930043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954503+0000","last_change":"2024-03-14T21:14:03.037873+0000","last_active":"2024-03-14T21:14:04.954503+0000","last_peered":"2024-03-14T21:14:04.954503+0000","last_clean":"2024-03-14T21:14:04.954503+0000","last_became_active":"2024-03-14T21:14:03.037608+0000","last_became_peered":"2024-03-14T21:14:03.037608+0000","last_unstale":"2024-03-14T21:14:04.954503+0000","last_undegraded":"2024-03-14T21:14:04.954503+0000","last_fullsized":"2024-03-14T21:14:04.954503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:46:32.269714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019235+0000","last_change":"2024-03-14T21:14:02.663747+0000","last_active":"2024-03-14T21:14:04.019235+0000","last_peered":"2024-03-14T21:14:04.019235+0000","last_clean":"2024-03-14T21:14:04.019235+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T21:14:04.019235+0000","last_undegraded":"2024-03-14T21:14:04.019235+0000","last_fullsized":"2024-03-14T21:14:04.019235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:59:11.091842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954684+0000","last_change":"2024-03-14T21:14:03.030867+0000","last_active":"2024-03-14T21:14:04.954684+0000","last_peered":"2024-03-14T21:14:04.954684+0000","last_clean":"2024-03-14T21:14:04.954684+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T21:14:04.954684+0000","last_undegraded":"2024-03-14T21:14:04.954684+0000","last_fullsized":"2024-03-14T21:14:04.954684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930134+0000","last_change":"2024-03-14T21:14:02.659095+0000","last_active":"2024-03-14T21:14:04.930134+0000","last_peered":"2024-03-14T21:14:04.930134+0000","last_clean":"2024-03-14T21:14:04.930134+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T21:14:04.930134+0000","last_undegraded":"2024-03-14T21:14:04.930134+0000","last_fullsized":"2024-03-14T21:14:04.930134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954730+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T21:14:04.954730+0000","last_peered":"2024-03-14T21:14:04.954730+0000","last_clean":"2024-03-14T21:14:04.954730+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T21:14:04.954730+0000","last_undegraded":"2024-03-14T21:14:04.954730+0000","last_fullsized":"2024-03-14T21:14:04.954730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017595+0000","last_change":"2024-03-14T21:14:02.663441+0000","last_active":"2024-03-14T21:14:04.017595+0000","last_peered":"2024-03-14T21:14:04.017595+0000","last_clean":"2024-03-14T21:14:04.017595+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T21:14:04.017595+0000","last_undegraded":"2024-03-14T21:14:04.017595+0000","last_fullsized":"2024-03-14T21:14:04.017595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.020037+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T21:14:04.020037+0000","last_peered":"2024-03-14T21:14:04.020037+0000","last_clean":"2024-03-14T21:14:04.020037+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T21:14:04.020037+0000","last_undegraded":"2024-03-14T21:14:04.020037+0000","last_fullsized":"2024-03-14T21:14:04.020037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930109+0000","last_change":"2024-03-14T21:14:02.653919+0000","last_active":"2024-03-14T21:14:04.930109+0000","last_peered":"2024-03-14T21:14:04.930109+0000","last_clean":"2024-03-14T21:14:04.930109+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T21:14:04.930109+0000","last_undegraded":"2024-03-14T21:14:04.930109+0000","last_fullsized":"2024-03-14T21:14:04.930109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954609+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T21:14:04.954609+0000","last_peered":"2024-03-14T21:14:04.954609+0000","last_clean":"2024-03-14T21:14:04.954609+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T21:14:04.954609+0000","last_undegraded":"2024-03-14T21:14:04.954609+0000","last_fullsized":"2024-03-14T21:14:04.954609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.021013+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T21:14:03.021013+0000","last_peered":"2024-03-14T21:14:03.021013+0000","last_clean":"2024-03-14T21:14:03.021013+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T21:14:03.021013+0000","last_undegraded":"2024-03-14T21:14:03.021013+0000","last_fullsized":"2024-03-14T21:14:03.021013+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019557+0000","last_change":"2024-03-14T21:14:02.652881+0000","last_active":"2024-03-14T21:14:04.019557+0000","last_peered":"2024-03-14T21:14:04.019557+0000","last_clean":"2024-03-14T21:14:04.019557+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T21:14:04.019557+0000","last_undegraded":"2024-03-14T21:14:04.019557+0000","last_fullsized":"2024-03-14T21:14:04.019557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243730+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T21:14:04.243730+0000","last_peered":"2024-03-14T21:14:04.243730+0000","last_clean":"2024-03-14T21:14:04.243730+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T21:14:04.243730+0000","last_undegraded":"2024-03-14T21:14:04.243730+0000","last_fullsized":"2024-03-14T21:14:04.243730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954460+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T21:14:04.954460+0000","last_peered":"2024-03-14T21:14:04.954460+0000","last_clean":"2024-03-14T21:14:04.954460+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T21:14:04.954460+0000","last_undegraded":"2024-03-14T21:14:04.954460+0000","last_fullsized":"2024-03-14T21:14:04.954460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:46.126855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928640+0000","last_change":"2024-03-14T21:14:03.033195+0000","last_active":"2024-03-14T21:14:04.928640+0000","last_peered":"2024-03-14T21:14:04.928640+0000","last_clean":"2024-03-14T21:14:04.928640+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T21:14:04.928640+0000","last_undegraded":"2024-03-14T21:14:04.928640+0000","last_fullsized":"2024-03-14T21:14:04.928640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930088+0000","last_change":"2024-03-14T21:14:02.658709+0000","last_active":"2024-03-14T21:14:04.930088+0000","last_peered":"2024-03-14T21:14:04.930088+0000","last_clean":"2024-03-14T21:14:04.930088+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T21:14:04.930088+0000","last_undegraded":"2024-03-14T21:14:04.930088+0000","last_fullsized":"2024-03-14T21:14:04.930088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017391+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T21:14:04.017391+0000","last_peered":"2024-03-14T21:14:04.017391+0000","last_clean":"2024-03-14T21:14:04.017391+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T21:14:04.017391+0000","last_undegraded":"2024-03-14T21:14:04.017391+0000","last_fullsized":"2024-03-14T21:14:04.017391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019554+0000","last_change":"2024-03-14T21:14:02.657401+0000","last_active":"2024-03-14T21:14:04.019554+0000","last_peered":"2024-03-14T21:14:04.019554+0000","last_clean":"2024-03-14T21:14:04.019554+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T21:14:04.019554+0000","last_undegraded":"2024-03-14T21:14:04.019554+0000","last_fullsized":"2024-03-14T21:14:04.019554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019342+0000","last_change":"2024-03-14T21:14:03.036152+0000","last_active":"2024-03-14T21:14:04.019342+0000","last_peered":"2024-03-14T21:14:04.019342+0000","last_clean":"2024-03-14T21:14:04.019342+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T21:14:04.019342+0000","last_undegraded":"2024-03-14T21:14:04.019342+0000","last_fullsized":"2024-03-14T21:14:04.019342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:14.794102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017565+0000","last_change":"2024-03-14T21:14:02.653065+0000","last_active":"2024-03-14T21:14:04.017565+0000","last_peered":"2024-03-14T21:14:04.017565+0000","last_clean":"2024-03-14T21:14:04.017565+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T21:14:04.017565+0000","last_undegraded":"2024-03-14T21:14:04.017565+0000","last_fullsized":"2024-03-14T21:14:04.017565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.020019+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T21:14:04.020019+0000","last_peered":"2024-03-14T21:14:04.020019+0000","last_clean":"2024-03-14T21:14:04.020019+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T21:14:04.020019+0000","last_undegraded":"2024-03-14T21:14:04.020019+0000","last_fullsized":"2024-03-14T21:14:04.020019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954668+0000","last_change":"2024-03-14T21:14:02.659612+0000","last_active":"2024-03-14T21:14:04.954668+0000","last_peered":"2024-03-14T21:14:04.954668+0000","last_clean":"2024-03-14T21:14:04.954668+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T21:14:04.954668+0000","last_undegraded":"2024-03-14T21:14:04.954668+0000","last_fullsized":"2024-03-14T21:14:04.954668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017470+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T21:14:04.017470+0000","last_peered":"2024-03-14T21:14:04.017470+0000","last_clean":"2024-03-14T21:14:04.017470+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T21:14:04.017470+0000","last_undegraded":"2024-03-14T21:14:04.017470+0000","last_fullsized":"2024-03-14T21:14:04.017470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.927982+0000","last_change":"2024-03-14T21:14:02.662122+0000","last_active":"2024-03-14T21:14:04.927982+0000","last_peered":"2024-03-14T21:14:04.927982+0000","last_clean":"2024-03-14T21:14:04.927982+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T21:14:04.927982+0000","last_undegraded":"2024-03-14T21:14:04.927982+0000","last_fullsized":"2024-03-14T21:14:04.927982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.244002+0000","last_change":"2024-03-14T21:14:03.037921+0000","last_active":"2024-03-14T21:14:04.244002+0000","last_peered":"2024-03-14T21:14:04.244002+0000","last_clean":"2024-03-14T21:14:04.244002+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T21:14:04.244002+0000","last_undegraded":"2024-03-14T21:14:04.244002+0000","last_fullsized":"2024-03-14T21:14:04.244002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928891+0000","last_change":"2024-03-14T21:14:02.665745+0000","last_active":"2024-03-14T21:14:04.928891+0000","last_peered":"2024-03-14T21:14:04.928891+0000","last_clean":"2024-03-14T21:14:04.928891+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T21:14:04.928891+0000","last_undegraded":"2024-03-14T21:14:04.928891+0000","last_fullsized":"2024-03-14T21:14:04.928891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928718+0000","last_change":"2024-03-14T21:14:03.029269+0000","last_active":"2024-03-14T21:14:04.928718+0000","last_peered":"2024-03-14T21:14:04.928718+0000","last_clean":"2024-03-14T21:14:04.928718+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T21:14:04.928718+0000","last_undegraded":"2024-03-14T21:14:04.928718+0000","last_fullsized":"2024-03-14T21:14:04.928718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954436+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T21:14:04.954436+0000","last_peered":"2024-03-14T21:14:04.954436+0000","last_clean":"2024-03-14T21:14:04.954436+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T21:14:04.954436+0000","last_undegraded":"2024-03-14T21:14:04.954436+0000","last_fullsized":"2024-03-14T21:14:04.954436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019008+0000","last_change":"2024-03-14T21:14:03.039549+0000","last_active":"2024-03-14T21:14:04.019008+0000","last_peered":"2024-03-14T21:14:04.019008+0000","last_clean":"2024-03-14T21:14:04.019008+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T21:14:04.019008+0000","last_undegraded":"2024-03-14T21:14:04.019008+0000","last_fullsized":"2024-03-14T21:14:04.019008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:23.510516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017601+0000","last_change":"2024-03-14T21:14:02.660292+0000","last_active":"2024-03-14T21:14:04.017601+0000","last_peered":"2024-03-14T21:14:04.017601+0000","last_clean":"2024-03-14T21:14:04.017601+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T21:14:04.017601+0000","last_undegraded":"2024-03-14T21:14:04.017601+0000","last_fullsized":"2024-03-14T21:14:04.017601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243693+0000","last_change":"2024-03-14T21:14:03.040817+0000","last_active":"2024-03-14T21:14:04.243693+0000","last_peered":"2024-03-14T21:14:04.243693+0000","last_clean":"2024-03-14T21:14:04.243693+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T21:14:04.243693+0000","last_undegraded":"2024-03-14T21:14:04.243693+0000","last_fullsized":"2024-03-14T21:14:04.243693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017553+0000","last_change":"2024-03-14T21:14:02.661073+0000","last_active":"2024-03-14T21:14:04.017553+0000","last_peered":"2024-03-14T21:14:04.017553+0000","last_clean":"2024-03-14T21:14:04.017553+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T21:14:04.017553+0000","last_undegraded":"2024-03-14T21:14:04.017553+0000","last_fullsized":"2024-03-14T21:14:04.017553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243996+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T21:14:04.243996+0000","last_peered":"2024-03-14T21:14:04.243996+0000","last_clean":"2024-03-14T21:14:04.243996+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T21:14:04.243996+0000","last_undegraded":"2024-03-14T21:14:04.243996+0000","last_fullsized":"2024-03-14T21:14:04.243996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954559+0000","last_change":"2024-03-14T21:14:02.663131+0000","last_active":"2024-03-14T21:14:04.954559+0000","last_peered":"2024-03-14T21:14:04.954559+0000","last_clean":"2024-03-14T21:14:04.954559+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T21:14:04.954559+0000","last_undegraded":"2024-03-14T21:14:04.954559+0000","last_fullsized":"2024-03-14T21:14:04.954559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928747+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T21:14:04.928747+0000","last_peered":"2024-03-14T21:14:04.928747+0000","last_clean":"2024-03-14T21:14:04.928747+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T21:14:04.928747+0000","last_undegraded":"2024-03-14T21:14:04.928747+0000","last_fullsized":"2024-03-14T21:14:04.928747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019966+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T21:14:04.019966+0000","last_peered":"2024-03-14T21:14:04.019966+0000","last_clean":"2024-03-14T21:14:04.019966+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T21:14:04.019966+0000","last_undegraded":"2024-03-14T21:14:04.019966+0000","last_fullsized":"2024-03-14T21:14:04.019966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954577+0000","last_change":"2024-03-14T21:14:02.660393+0000","last_active":"2024-03-14T21:14:04.954577+0000","last_peered":"2024-03-14T21:14:04.954577+0000","last_clean":"2024-03-14T21:14:04.954577+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T21:14:04.954577+0000","last_undegraded":"2024-03-14T21:14:04.954577+0000","last_fullsized":"2024-03-14T21:14:04.954577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019982+0000","last_change":"2024-03-14T21:14:03.035149+0000","last_active":"2024-03-14T21:14:04.019982+0000","last_peered":"2024-03-14T21:14:04.019982+0000","last_clean":"2024-03-14T21:14:04.019982+0000","last_became_active":"2024-03-14T21:14:03.034649+0000","last_became_peered":"2024-03-14T21:14:03.034649+0000","last_unstale":"2024-03-14T21:14:04.019982+0000","last_undegraded":"2024-03-14T21:14:04.019982+0000","last_fullsized":"2024-03-14T21:14:04.019982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954625+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T21:14:04.954625+0000","last_peered":"2024-03-14T21:14:04.954625+0000","last_clean":"2024-03-14T21:14:04.954625+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T21:14:04.954625+0000","last_undegraded":"2024-03-14T21:14:04.954625+0000","last_fullsized":"2024-03-14T21:14:04.954625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:48.203059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928726+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T21:14:04.928726+0000","last_peered":"2024-03-14T21:14:04.928726+0000","last_clean":"2024-03-14T21:14:04.928726+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T21:14:04.928726+0000","last_undegraded":"2024-03-14T21:14:04.928726+0000","last_fullsized":"2024-03-14T21:14:04.928726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243988+0000","last_change":"2024-03-14T21:14:02.654963+0000","last_active":"2024-03-14T21:14:04.243988+0000","last_peered":"2024-03-14T21:14:04.243988+0000","last_clean":"2024-03-14T21:14:04.243988+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T21:14:04.243988+0000","last_undegraded":"2024-03-14T21:14:04.243988+0000","last_fullsized":"2024-03-14T21:14:04.243988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243950+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T21:14:04.243950+0000","last_peered":"2024-03-14T21:14:04.243950+0000","last_clean":"2024-03-14T21:14:04.243950+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T21:14:04.243950+0000","last_undegraded":"2024-03-14T21:14:04.243950+0000","last_fullsized":"2024-03-14T21:14:04.243950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928743+0000","last_change":"2024-03-14T21:14:02.656125+0000","last_active":"2024-03-14T21:14:04.928743+0000","last_peered":"2024-03-14T21:14:04.928743+0000","last_clean":"2024-03-14T21:14:04.928743+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T21:14:04.928743+0000","last_undegraded":"2024-03-14T21:14:04.928743+0000","last_fullsized":"2024-03-14T21:14:04.928743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":819200,"data_stored":665956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T21:14:11.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T21:14:11.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.232+0000 7fc9e8cbd640 1 Processor -- start 2024-03-14T21:14:11.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.232+0000 7fc9e8cbd640 1 -- start start 2024-03-14T21:14:11.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e8cbd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9e406b620 0x7fc9e40bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e8cbd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e406a6e0 con 0x7fc9e40baef0 2024-03-14T21:14:11.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e8cbd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40bc2b0 con 0x7fc9e40bc940 2024-03-14T21:14:11.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e8cbd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e406ad10 con 0x7fc9e406b620 2024-03-14T21:14:11.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e259c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc9e40baef0 0x7fc9e41480b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46950/0 (socket says 172.21.15.49:46950) 2024-03-14T21:14:11.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e259c640 1 -- 172.21.15.49:0/4142849819 learned_addr learned my addr 172.21.15.49:0/4142849819 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:11.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e1d9b640 1 --2- 172.21.15.49:0/4142849819 >> v2:172.21.15.97:3300/0 conn(0x7fc9e406b620 0x7fc9e40bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137903567 0 0) 0x7fc9e406a6e0 con 0x7fc9e40baef0 2024-03-14T21:14:11.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d0003520 con 0x7fc9e40baef0 2024-03-14T21:14:11.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689552313 0 0) 0x7fc9e40bc2b0 con 0x7fc9e40bc940 2024-03-14T21:14:11.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e406a6e0 con 0x7fc9e40bc940 2024-03-14T21:14:11.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e1d9b640 1 -- 172.21.15.49:0/4142849819 >> v1:172.21.15.97:6789/0 conn(0x7fc9e40bc940 legacy=0x7fc9e4155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e1d9b640 1 -- 172.21.15.49:0/4142849819 >> v1:172.21.15.49:6789/0 conn(0x7fc9e40baef0 legacy=0x7fc9e41480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.233+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2097893117 0 0) 0x7fc9d0003520 con 0x7fc9e40baef0 2024-03-14T21:14:11.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.234+0000 7fc9e1d9b640 1 -- 172.21.15.49:0/4142849819 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e406bab0 con 0x7fc9e406b620 2024-03-14T21:14:11.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.234+0000 7fc9e1d9b640 1 --2- 172.21.15.49:0/4142849819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e406b620 0x7fc9e40bf7f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0097a0 tx=0x7fc9cc02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6515844d3e7b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.234+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc001670 con 0x7fc9e406b620 2024-03-14T21:14:11.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.234+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc02e050 con 0x7fc9e406b620 2024-03-14T21:14:11.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.234+0000 7fc9e159a640 1 -- 172.21.15.49:0/4142849819 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc030710 con 0x7fc9e406b620 2024-03-14T21:14:11.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.235+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/4142849819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e406b620 msgr2=0x7fc9e40bf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.235+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/4142849819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e406b620 0x7fc9e40bf7f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0097a0 tx=0x7fc9cc02b790 comp rx=0 tx=0).stop 2024-03-14T21:14:11.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.235+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/4142849819 shutdown_connections 2024-03-14T21:14:11.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.235+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/4142849819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e406b620 0x7fc9e40bf7f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.235+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/4142849819 >> 172.21.15.49:0/4142849819 conn(0x7fc9e40bcd80 msgr2=0x7fc9e40bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:11.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.235+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/4142849819 shutdown_connections 2024-03-14T21:14:11.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.235+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/4142849819 wait complete. 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.236+0000 7fc9e8cbd640 1 Processor -- start 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.236+0000 7fc9e8cbd640 1 -- start start 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e8cbd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9e40baef0 0x7fc9e41d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e8cbd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9e40bc940 0x7fc9e40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e8cbd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e40b4d20 0x7fc9e41dfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e8cbd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e41d16a0 con 0x7fc9e40bc940 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e8cbd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40a2ea0 con 0x7fc9e40b4d20 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e8cbd640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b37a0 con 0x7fc9e40baef0 2024-03-14T21:14:11.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9e40bc940 0x7fc9e40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9e40bc940 0x7fc9e40b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43792/0 (socket says 172.21.15.49:43792) 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 -- 172.21.15.49:0/577047754 learned_addr learned my addr 172.21.15.49:0/577047754 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e259c640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9e40baef0 0x7fc9e41d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e2d9d640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e40b4d20 0x7fc9e41dfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 -- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9e40baef0 msgr2=0x7fc9e41d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9e40baef0 0x7fc9e41d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 -- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e40b4d20 msgr2=0x7fc9e41dfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e40b4d20 0x7fc9e41dfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.237+0000 7fc9e1d9b640 1 -- 172.21.15.49:0/577047754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc009400 con 0x7fc9e40bc940 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.238+0000 7fc9e1d9b640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9e40bc940 0x7fc9e40b47e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098c0 tx=0x7fc9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.238+0000 7fc9daffd640 1 -- 172.21.15.49:0/577047754 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc03c620 con 0x7fc9e40bc940 2024-03-14T21:14:11.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.238+0000 7fc9daffd640 1 -- 172.21.15.49:0/577047754 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc03c7a0 con 0x7fc9e40bc940 2024-03-14T21:14:11.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.238+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e4148940 con 0x7fc9e40bc940 2024-03-14T21:14:11.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.238+0000 7fc9daffd640 1 -- 172.21.15.49:0/577047754 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc03cb50 con 0x7fc9e40bc940 2024-03-14T21:14:11.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.238+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e40b3ac0 con 0x7fc9e40bc940 2024-03-14T21:14:11.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.239+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40b38e0 con 0x7fc9e40bc940 2024-03-14T21:14:11.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.239+0000 7fc9daffd640 1 -- 172.21.15.49:0/577047754 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc9cc039070 con 0x7fc9e40bc940 2024-03-14T21:14:11.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.240+0000 7fc9daffd640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc9c406a7a0 0x7fc9c406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.240+0000 7fc9daffd640 1 -- 172.21.15.49:0/577047754 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9cc0aa380 con 0x7fc9e40bc940 2024-03-14T21:14:11.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.240+0000 7fc9e259c640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc9c406a7a0 0x7fc9c406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.240+0000 7fc9e259c640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc9c406a7a0 0x7fc9c406cc60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc000fd0 tx=0x7fc9dc001690 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.244+0000 7fc9daffd640 1 -- 172.21.15.49:0/577047754 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc9cc0758e0 con 0x7fc9e40bc940 2024-03-14T21:14:11.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.418+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9e40bb2d0 con 0x7fc9c406a7a0 2024-03-14T21:14:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.425+0000 7fc9daffd640 1 -- 172.21.15.49:0/577047754 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264577 (secure 0 0 0) 0x7fc9dc00f940 con 0x7fc9c406a7a0 2024-03-14T21:14:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc9c406a7a0 msgr2=0x7fc9c406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc9c406a7a0 0x7fc9c406cc60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc000fd0 tx=0x7fc9dc001690 comp rx=0 tx=0).stop 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9e40bc940 msgr2=0x7fc9e40b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9e40bc940 0x7fc9e40b47e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098c0 tx=0x7fc9cc0046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 shutdown_connections 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc9c406a7a0 0x7fc9c406cc60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e40b4d20 0x7fc9e41dfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9e40bc940 0x7fc9e40b47e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 --2- 172.21.15.49:0/577047754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9e40baef0 0x7fc9e41d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 >> 172.21.15.49:0/577047754 conn(0x7fc9e40bcd80 msgr2=0x7fc9e4146cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 shutdown_connections 2024-03-14T21:14:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.428+0000 7fc9e8cbd640 1 -- 172.21.15.49:0/577047754 wait complete. 2024-03-14T21:14:11.422 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-03-14T21:14:10.911083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235460,"kb_used_data":2916,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754739260,"statfs":{"total":773094113280,"available":772853002240,"internally_reserved":0,"allocated":2985984,"data_stored":1789393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002341"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954700+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T21:14:04.954700+0000","last_peered":"2024-03-14T21:14:04.954700+0000","last_clean":"2024-03-14T21:14:04.954700+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T21:14:04.954700+0000","last_undegraded":"2024-03-14T21:14:04.954700+0000","last_fullsized":"2024-03-14T21:14:04.954700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:57:09.325211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017662+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T21:14:04.017662+0000","last_peered":"2024-03-14T21:14:04.017662+0000","last_clean":"2024-03-14T21:14:04.017662+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T21:14:04.017662+0000","last_undegraded":"2024-03-14T21:14:04.017662+0000","last_fullsized":"2024-03-14T21:14:04.017662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928709+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T21:14:04.928709+0000","last_peered":"2024-03-14T21:14:04.928709+0000","last_clean":"2024-03-14T21:14:04.928709+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T21:14:04.928709+0000","last_undegraded":"2024-03-14T21:14:04.928709+0000","last_fullsized":"2024-03-14T21:14:04.928709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:08:55.492956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019894+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T21:14:04.019894+0000","last_peered":"2024-03-14T21:14:04.019894+0000","last_clean":"2024-03-14T21:14:04.019894+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T21:14:04.019894+0000","last_undegraded":"2024-03-14T21:14:04.019894+0000","last_fullsized":"2024-03-14T21:14:04.019894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954528+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T21:14:04.954528+0000","last_peered":"2024-03-14T21:14:04.954528+0000","last_clean":"2024-03-14T21:14:04.954528+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T21:14:04.954528+0000","last_undegraded":"2024-03-14T21:14:04.954528+0000","last_fullsized":"2024-03-14T21:14:04.954528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019354+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T21:14:04.019354+0000","last_peered":"2024-03-14T21:14:04.019354+0000","last_clean":"2024-03-14T21:14:04.019354+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T21:14:04.019354+0000","last_undegraded":"2024-03-14T21:14:04.019354+0000","last_fullsized":"2024-03-14T21:14:04.019354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954494+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T21:14:04.954494+0000","last_peered":"2024-03-14T21:14:04.954494+0000","last_clean":"2024-03-14T21:14:04.954494+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T21:14:04.954494+0000","last_undegraded":"2024-03-14T21:14:04.954494+0000","last_fullsized":"2024-03-14T21:14:04.954494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243611+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T21:14:04.243611+0000","last_peered":"2024-03-14T21:14:04.243611+0000","last_clean":"2024-03-14T21:14:04.243611+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T21:14:04.243611+0000","last_undegraded":"2024-03-14T21:14:04.243611+0000","last_fullsized":"2024-03-14T21:14:04.243611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954715+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T21:14:04.954715+0000","last_peered":"2024-03-14T21:14:04.954715+0000","last_clean":"2024-03-14T21:14:04.954715+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T21:14:04.954715+0000","last_undegraded":"2024-03-14T21:14:04.954715+0000","last_fullsized":"2024-03-14T21:14:04.954715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928838+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T21:14:04.928838+0000","last_peered":"2024-03-14T21:14:04.928838+0000","last_clean":"2024-03-14T21:14:04.928838+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T21:14:04.928838+0000","last_undegraded":"2024-03-14T21:14:04.928838+0000","last_fullsized":"2024-03-14T21:14:04.928838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928822+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T21:14:04.928822+0000","last_peered":"2024-03-14T21:14:04.928822+0000","last_clean":"2024-03-14T21:14:04.928822+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T21:14:04.928822+0000","last_undegraded":"2024-03-14T21:14:04.928822+0000","last_fullsized":"2024-03-14T21:14:04.928822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019337+0000","last_change":"2024-03-14T21:14:03.034043+0000","last_active":"2024-03-14T21:14:04.019337+0000","last_peered":"2024-03-14T21:14:04.019337+0000","last_clean":"2024-03-14T21:14:04.019337+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T21:14:04.019337+0000","last_undegraded":"2024-03-14T21:14:04.019337+0000","last_fullsized":"2024-03-14T21:14:04.019337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:33.787920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243838+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T21:14:04.243838+0000","last_peered":"2024-03-14T21:14:04.243838+0000","last_clean":"2024-03-14T21:14:04.243838+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T21:14:04.243838+0000","last_undegraded":"2024-03-14T21:14:04.243838+0000","last_fullsized":"2024-03-14T21:14:04.243838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928654+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T21:14:04.928654+0000","last_peered":"2024-03-14T21:14:04.928654+0000","last_clean":"2024-03-14T21:14:04.928654+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T21:14:04.928654+0000","last_undegraded":"2024-03-14T21:14:04.928654+0000","last_fullsized":"2024-03-14T21:14:04.928654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017669+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T21:14:04.017669+0000","last_peered":"2024-03-14T21:14:04.017669+0000","last_clean":"2024-03-14T21:14:04.017669+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T21:14:04.017669+0000","last_undegraded":"2024-03-14T21:14:04.017669+0000","last_fullsized":"2024-03-14T21:14:04.017669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018774+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T21:14:04.018774+0000","last_peered":"2024-03-14T21:14:04.018774+0000","last_clean":"2024-03-14T21:14:04.018774+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T21:14:04.018774+0000","last_undegraded":"2024-03-14T21:14:04.018774+0000","last_fullsized":"2024-03-14T21:14:04.018774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019994+0000","last_change":"2024-03-14T21:14:03.035328+0000","last_active":"2024-03-14T21:14:04.019994+0000","last_peered":"2024-03-14T21:14:04.019994+0000","last_clean":"2024-03-14T21:14:04.019994+0000","last_became_active":"2024-03-14T21:14:03.034934+0000","last_became_peered":"2024-03-14T21:14:03.034934+0000","last_unstale":"2024-03-14T21:14:04.019994+0000","last_undegraded":"2024-03-14T21:14:04.019994+0000","last_fullsized":"2024-03-14T21:14:04.019994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930124+0000","last_change":"2024-03-14T21:14:03.029823+0000","last_active":"2024-03-14T21:14:04.930124+0000","last_peered":"2024-03-14T21:14:04.930124+0000","last_clean":"2024-03-14T21:14:04.930124+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T21:14:04.930124+0000","last_undegraded":"2024-03-14T21:14:04.930124+0000","last_fullsized":"2024-03-14T21:14:04.930124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:47.131679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954641+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T21:14:04.954641+0000","last_peered":"2024-03-14T21:14:04.954641+0000","last_clean":"2024-03-14T21:14:04.954641+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T21:14:04.954641+0000","last_undegraded":"2024-03-14T21:14:04.954641+0000","last_fullsized":"2024-03-14T21:14:04.954641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928783+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T21:14:04.928783+0000","last_peered":"2024-03-14T21:14:04.928783+0000","last_clean":"2024-03-14T21:14:04.928783+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T21:14:04.928783+0000","last_undegraded":"2024-03-14T21:14:04.928783+0000","last_fullsized":"2024-03-14T21:14:04.928783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928673+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T21:14:04.928673+0000","last_peered":"2024-03-14T21:14:04.928673+0000","last_clean":"2024-03-14T21:14:04.928673+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T21:14:04.928673+0000","last_undegraded":"2024-03-14T21:14:04.928673+0000","last_fullsized":"2024-03-14T21:14:04.928673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019386+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T21:14:04.019386+0000","last_peered":"2024-03-14T21:14:04.019386+0000","last_clean":"2024-03-14T21:14:04.019386+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T21:14:04.019386+0000","last_undegraded":"2024-03-14T21:14:04.019386+0000","last_fullsized":"2024-03-14T21:14:04.019386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243662+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T21:14:04.243662+0000","last_peered":"2024-03-14T21:14:04.243662+0000","last_clean":"2024-03-14T21:14:04.243662+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T21:14:04.243662+0000","last_undegraded":"2024-03-14T21:14:04.243662+0000","last_fullsized":"2024-03-14T21:14:04.243662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018867+0000","last_change":"2024-03-14T21:14:03.038694+0000","last_active":"2024-03-14T21:14:04.018867+0000","last_peered":"2024-03-14T21:14:04.018867+0000","last_clean":"2024-03-14T21:14:04.018867+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T21:14:04.018867+0000","last_undegraded":"2024-03-14T21:14:04.018867+0000","last_fullsized":"2024-03-14T21:14:04.018867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243990+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T21:14:04.243990+0000","last_peered":"2024-03-14T21:14:04.243990+0000","last_clean":"2024-03-14T21:14:04.243990+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T21:14:04.243990+0000","last_undegraded":"2024-03-14T21:14:04.243990+0000","last_fullsized":"2024-03-14T21:14:04.243990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928859+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T21:14:04.928859+0000","last_peered":"2024-03-14T21:14:04.928859+0000","last_clean":"2024-03-14T21:14:04.928859+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T21:14:04.928859+0000","last_undegraded":"2024-03-14T21:14:04.928859+0000","last_fullsized":"2024-03-14T21:14:04.928859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017498+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T21:14:04.017498+0000","last_peered":"2024-03-14T21:14:04.017498+0000","last_clean":"2024-03-14T21:14:04.017498+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T21:14:04.017498+0000","last_undegraded":"2024-03-14T21:14:04.017498+0000","last_fullsized":"2024-03-14T21:14:04.017498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243915+0000","last_change":"2024-03-14T21:14:03.040690+0000","last_active":"2024-03-14T21:14:04.243915+0000","last_peered":"2024-03-14T21:14:04.243915+0000","last_clean":"2024-03-14T21:14:04.243915+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T21:14:04.243915+0000","last_undegraded":"2024-03-14T21:14:04.243915+0000","last_fullsized":"2024-03-14T21:14:04.243915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928682+0000","last_change":"2024-03-14T21:14:03.038048+0000","last_active":"2024-03-14T21:14:04.928682+0000","last_peered":"2024-03-14T21:14:04.928682+0000","last_clean":"2024-03-14T21:14:04.928682+0000","last_became_active":"2024-03-14T21:14:03.037273+0000","last_became_peered":"2024-03-14T21:14:03.037273+0000","last_unstale":"2024-03-14T21:14:04.928682+0000","last_undegraded":"2024-03-14T21:14:04.928682+0000","last_fullsized":"2024-03-14T21:14:04.928682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019469+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T21:14:04.019469+0000","last_peered":"2024-03-14T21:14:04.019469+0000","last_clean":"2024-03-14T21:14:04.019469+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T21:14:04.019469+0000","last_undegraded":"2024-03-14T21:14:04.019469+0000","last_fullsized":"2024-03-14T21:14:04.019469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930068+0000","last_change":"2024-03-14T21:14:03.029847+0000","last_active":"2024-03-14T21:14:04.930068+0000","last_peered":"2024-03-14T21:14:04.930068+0000","last_clean":"2024-03-14T21:14:04.930068+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T21:14:04.930068+0000","last_undegraded":"2024-03-14T21:14:04.930068+0000","last_fullsized":"2024-03-14T21:14:04.930068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928688+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T21:14:04.928688+0000","last_peered":"2024-03-14T21:14:04.928688+0000","last_clean":"2024-03-14T21:14:04.928688+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T21:14:04.928688+0000","last_undegraded":"2024-03-14T21:14:04.928688+0000","last_fullsized":"2024-03-14T21:14:04.928688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018978+0000","last_change":"2024-03-14T21:14:03.037915+0000","last_active":"2024-03-14T21:14:04.018978+0000","last_peered":"2024-03-14T21:14:04.018978+0000","last_clean":"2024-03-14T21:14:04.018978+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T21:14:04.018978+0000","last_undegraded":"2024-03-14T21:14:04.018978+0000","last_fullsized":"2024-03-14T21:14:04.018978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243993+0000","last_change":"2024-03-14T21:14:02.655762+0000","last_active":"2024-03-14T21:14:04.243993+0000","last_peered":"2024-03-14T21:14:04.243993+0000","last_clean":"2024-03-14T21:14:04.243993+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T21:14:04.243993+0000","last_undegraded":"2024-03-14T21:14:04.243993+0000","last_fullsized":"2024-03-14T21:14:04.243993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928690+0000","last_change":"2024-03-14T21:14:03.033091+0000","last_active":"2024-03-14T21:14:04.928690+0000","last_peered":"2024-03-14T21:14:04.928690+0000","last_clean":"2024-03-14T21:14:04.928690+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T21:14:04.928690+0000","last_undegraded":"2024-03-14T21:14:04.928690+0000","last_fullsized":"2024-03-14T21:14:04.928690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928765+0000","last_change":"2024-03-14T21:14:02.662899+0000","last_active":"2024-03-14T21:14:04.928765+0000","last_peered":"2024-03-14T21:14:04.928765+0000","last_clean":"2024-03-14T21:14:04.928765+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T21:14:04.928765+0000","last_undegraded":"2024-03-14T21:14:04.928765+0000","last_fullsized":"2024-03-14T21:14:04.928765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243862+0000","last_change":"2024-03-14T21:14:02.660574+0000","last_active":"2024-03-14T21:14:04.243862+0000","last_peered":"2024-03-14T21:14:04.243862+0000","last_clean":"2024-03-14T21:14:04.243862+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T21:14:04.243862+0000","last_undegraded":"2024-03-14T21:14:04.243862+0000","last_fullsized":"2024-03-14T21:14:04.243862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243590+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T21:14:04.243590+0000","last_peered":"2024-03-14T21:14:04.243590+0000","last_clean":"2024-03-14T21:14:04.243590+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T21:14:04.243590+0000","last_undegraded":"2024-03-14T21:14:04.243590+0000","last_fullsized":"2024-03-14T21:14:04.243590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.018961+0000","last_change":"2024-03-14T21:14:02.661303+0000","last_active":"2024-03-14T21:14:04.018961+0000","last_peered":"2024-03-14T21:14:04.018961+0000","last_clean":"2024-03-14T21:14:04.018961+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T21:14:04.018961+0000","last_undegraded":"2024-03-14T21:14:04.018961+0000","last_fullsized":"2024-03-14T21:14:04.018961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017394+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T21:14:04.017394+0000","last_peered":"2024-03-14T21:14:04.017394+0000","last_clean":"2024-03-14T21:14:04.017394+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T21:14:04.017394+0000","last_undegraded":"2024-03-14T21:14:04.017394+0000","last_fullsized":"2024-03-14T21:14:04.017394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928875+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T21:14:04.928875+0000","last_peered":"2024-03-14T21:14:04.928875+0000","last_clean":"2024-03-14T21:14:04.928875+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T21:14:04.928875+0000","last_undegraded":"2024-03-14T21:14:04.928875+0000","last_fullsized":"2024-03-14T21:14:04.928875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019922+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T21:14:04.019922+0000","last_peered":"2024-03-14T21:14:04.019922+0000","last_clean":"2024-03-14T21:14:04.019922+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T21:14:04.019922+0000","last_undegraded":"2024-03-14T21:14:04.019922+0000","last_fullsized":"2024-03-14T21:14:04.019922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019816+0000","last_change":"2024-03-14T21:14:02.660384+0000","last_active":"2024-03-14T21:14:04.019816+0000","last_peered":"2024-03-14T21:14:04.019816+0000","last_clean":"2024-03-14T21:14:04.019816+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T21:14:04.019816+0000","last_undegraded":"2024-03-14T21:14:04.019816+0000","last_fullsized":"2024-03-14T21:14:04.019816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928734+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T21:14:04.928734+0000","last_peered":"2024-03-14T21:14:04.928734+0000","last_clean":"2024-03-14T21:14:04.928734+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T21:14:04.928734+0000","last_undegraded":"2024-03-14T21:14:04.928734+0000","last_fullsized":"2024-03-14T21:14:04.928734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:57:33.155885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017568+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T21:14:04.017568+0000","last_peered":"2024-03-14T21:14:04.017568+0000","last_clean":"2024-03-14T21:14:04.017568+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T21:14:04.017568+0000","last_undegraded":"2024-03-14T21:14:04.017568+0000","last_fullsized":"2024-03-14T21:14:04.017568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019326+0000","last_change":"2024-03-14T21:14:03.032533+0000","last_active":"2024-03-14T21:14:04.019326+0000","last_peered":"2024-03-14T21:14:04.019326+0000","last_clean":"2024-03-14T21:14:04.019326+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T21:14:04.019326+0000","last_undegraded":"2024-03-14T21:14:04.019326+0000","last_fullsized":"2024-03-14T21:14:04.019326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:41.581386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928805+0000","last_change":"2024-03-14T21:14:02.659028+0000","last_active":"2024-03-14T21:14:04.928805+0000","last_peered":"2024-03-14T21:14:04.928805+0000","last_clean":"2024-03-14T21:14:04.928805+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T21:14:04.928805+0000","last_undegraded":"2024-03-14T21:14:04.928805+0000","last_fullsized":"2024-03-14T21:14:04.928805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954595+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T21:14:04.954595+0000","last_peered":"2024-03-14T21:14:04.954595+0000","last_clean":"2024-03-14T21:14:04.954595+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T21:14:04.954595+0000","last_undegraded":"2024-03-14T21:14:04.954595+0000","last_fullsized":"2024-03-14T21:14:04.954595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928699+0000","last_change":"2024-03-14T21:14:02.663139+0000","last_active":"2024-03-14T21:14:04.928699+0000","last_peered":"2024-03-14T21:14:04.928699+0000","last_clean":"2024-03-14T21:14:04.928699+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T21:14:04.928699+0000","last_undegraded":"2024-03-14T21:14:04.928699+0000","last_fullsized":"2024-03-14T21:14:04.928699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:25:31.095452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928664+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T21:14:04.928664+0000","last_peered":"2024-03-14T21:14:04.928664+0000","last_clean":"2024-03-14T21:14:04.928664+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T21:14:04.928664+0000","last_undegraded":"2024-03-14T21:14:04.928664+0000","last_fullsized":"2024-03-14T21:14:04.928664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019604+0000","last_change":"2024-03-14T21:14:02.659773+0000","last_active":"2024-03-14T21:14:04.019604+0000","last_peered":"2024-03-14T21:14:04.019604+0000","last_clean":"2024-03-14T21:14:04.019604+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T21:14:04.019604+0000","last_undegraded":"2024-03-14T21:14:04.019604+0000","last_fullsized":"2024-03-14T21:14:04.019604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954543+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T21:14:04.954543+0000","last_peered":"2024-03-14T21:14:04.954543+0000","last_clean":"2024-03-14T21:14:04.954543+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T21:14:04.954543+0000","last_undegraded":"2024-03-14T21:14:04.954543+0000","last_fullsized":"2024-03-14T21:14:04.954543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928731+0000","last_change":"2024-03-14T21:14:02.663585+0000","last_active":"2024-03-14T21:14:04.928731+0000","last_peered":"2024-03-14T21:14:04.928731+0000","last_clean":"2024-03-14T21:14:04.928731+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T21:14:04.928731+0000","last_undegraded":"2024-03-14T21:14:04.928731+0000","last_fullsized":"2024-03-14T21:14:04.928731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928713+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T21:14:04.928713+0000","last_peered":"2024-03-14T21:14:04.928713+0000","last_clean":"2024-03-14T21:14:04.928713+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T21:14:04.928713+0000","last_undegraded":"2024-03-14T21:14:04.928713+0000","last_fullsized":"2024-03-14T21:14:04.928713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930043+0000","last_change":"2024-03-14T21:14:02.654931+0000","last_active":"2024-03-14T21:14:04.930043+0000","last_peered":"2024-03-14T21:14:04.930043+0000","last_clean":"2024-03-14T21:14:04.930043+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T21:14:04.930043+0000","last_undegraded":"2024-03-14T21:14:04.930043+0000","last_fullsized":"2024-03-14T21:14:04.930043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954503+0000","last_change":"2024-03-14T21:14:03.037873+0000","last_active":"2024-03-14T21:14:04.954503+0000","last_peered":"2024-03-14T21:14:04.954503+0000","last_clean":"2024-03-14T21:14:04.954503+0000","last_became_active":"2024-03-14T21:14:03.037608+0000","last_became_peered":"2024-03-14T21:14:03.037608+0000","last_unstale":"2024-03-14T21:14:04.954503+0000","last_undegraded":"2024-03-14T21:14:04.954503+0000","last_fullsized":"2024-03-14T21:14:04.954503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:46:32.269714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019235+0000","last_change":"2024-03-14T21:14:02.663747+0000","last_active":"2024-03-14T21:14:04.019235+0000","last_peered":"2024-03-14T21:14:04.019235+0000","last_clean":"2024-03-14T21:14:04.019235+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T21:14:04.019235+0000","last_undegraded":"2024-03-14T21:14:04.019235+0000","last_fullsized":"2024-03-14T21:14:04.019235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:59:11.091842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954684+0000","last_change":"2024-03-14T21:14:03.030867+0000","last_active":"2024-03-14T21:14:04.954684+0000","last_peered":"2024-03-14T21:14:04.954684+0000","last_clean":"2024-03-14T21:14:04.954684+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T21:14:04.954684+0000","last_undegraded":"2024-03-14T21:14:04.954684+0000","last_fullsized":"2024-03-14T21:14:04.954684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930134+0000","last_change":"2024-03-14T21:14:02.659095+0000","last_active":"2024-03-14T21:14:04.930134+0000","last_peered":"2024-03-14T21:14:04.930134+0000","last_clean":"2024-03-14T21:14:04.930134+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T21:14:04.930134+0000","last_undegraded":"2024-03-14T21:14:04.930134+0000","last_fullsized":"2024-03-14T21:14:04.930134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954730+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T21:14:04.954730+0000","last_peered":"2024-03-14T21:14:04.954730+0000","last_clean":"2024-03-14T21:14:04.954730+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T21:14:04.954730+0000","last_undegraded":"2024-03-14T21:14:04.954730+0000","last_fullsized":"2024-03-14T21:14:04.954730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017595+0000","last_change":"2024-03-14T21:14:02.663441+0000","last_active":"2024-03-14T21:14:04.017595+0000","last_peered":"2024-03-14T21:14:04.017595+0000","last_clean":"2024-03-14T21:14:04.017595+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T21:14:04.017595+0000","last_undegraded":"2024-03-14T21:14:04.017595+0000","last_fullsized":"2024-03-14T21:14:04.017595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.020037+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T21:14:04.020037+0000","last_peered":"2024-03-14T21:14:04.020037+0000","last_clean":"2024-03-14T21:14:04.020037+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T21:14:04.020037+0000","last_undegraded":"2024-03-14T21:14:04.020037+0000","last_fullsized":"2024-03-14T21:14:04.020037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930109+0000","last_change":"2024-03-14T21:14:02.653919+0000","last_active":"2024-03-14T21:14:04.930109+0000","last_peered":"2024-03-14T21:14:04.930109+0000","last_clean":"2024-03-14T21:14:04.930109+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T21:14:04.930109+0000","last_undegraded":"2024-03-14T21:14:04.930109+0000","last_fullsized":"2024-03-14T21:14:04.930109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954609+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T21:14:04.954609+0000","last_peered":"2024-03-14T21:14:04.954609+0000","last_clean":"2024-03-14T21:14:04.954609+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T21:14:04.954609+0000","last_undegraded":"2024-03-14T21:14:04.954609+0000","last_fullsized":"2024-03-14T21:14:04.954609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:03.021013+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T21:14:03.021013+0000","last_peered":"2024-03-14T21:14:03.021013+0000","last_clean":"2024-03-14T21:14:03.021013+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T21:14:03.021013+0000","last_undegraded":"2024-03-14T21:14:03.021013+0000","last_fullsized":"2024-03-14T21:14:03.021013+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019557+0000","last_change":"2024-03-14T21:14:02.652881+0000","last_active":"2024-03-14T21:14:04.019557+0000","last_peered":"2024-03-14T21:14:04.019557+0000","last_clean":"2024-03-14T21:14:04.019557+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T21:14:04.019557+0000","last_undegraded":"2024-03-14T21:14:04.019557+0000","last_fullsized":"2024-03-14T21:14:04.019557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243730+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T21:14:04.243730+0000","last_peered":"2024-03-14T21:14:04.243730+0000","last_clean":"2024-03-14T21:14:04.243730+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T21:14:04.243730+0000","last_undegraded":"2024-03-14T21:14:04.243730+0000","last_fullsized":"2024-03-14T21:14:04.243730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954460+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T21:14:04.954460+0000","last_peered":"2024-03-14T21:14:04.954460+0000","last_clean":"2024-03-14T21:14:04.954460+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T21:14:04.954460+0000","last_undegraded":"2024-03-14T21:14:04.954460+0000","last_fullsized":"2024-03-14T21:14:04.954460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:46.126855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928640+0000","last_change":"2024-03-14T21:14:03.033195+0000","last_active":"2024-03-14T21:14:04.928640+0000","last_peered":"2024-03-14T21:14:04.928640+0000","last_clean":"2024-03-14T21:14:04.928640+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T21:14:04.928640+0000","last_undegraded":"2024-03-14T21:14:04.928640+0000","last_fullsized":"2024-03-14T21:14:04.928640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.930088+0000","last_change":"2024-03-14T21:14:02.658709+0000","last_active":"2024-03-14T21:14:04.930088+0000","last_peered":"2024-03-14T21:14:04.930088+0000","last_clean":"2024-03-14T21:14:04.930088+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T21:14:04.930088+0000","last_undegraded":"2024-03-14T21:14:04.930088+0000","last_fullsized":"2024-03-14T21:14:04.930088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017391+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T21:14:04.017391+0000","last_peered":"2024-03-14T21:14:04.017391+0000","last_clean":"2024-03-14T21:14:04.017391+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T21:14:04.017391+0000","last_undegraded":"2024-03-14T21:14:04.017391+0000","last_fullsized":"2024-03-14T21:14:04.017391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019554+0000","last_change":"2024-03-14T21:14:02.657401+0000","last_active":"2024-03-14T21:14:04.019554+0000","last_peered":"2024-03-14T21:14:04.019554+0000","last_clean":"2024-03-14T21:14:04.019554+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T21:14:04.019554+0000","last_undegraded":"2024-03-14T21:14:04.019554+0000","last_fullsized":"2024-03-14T21:14:04.019554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019342+0000","last_change":"2024-03-14T21:14:03.036152+0000","last_active":"2024-03-14T21:14:04.019342+0000","last_peered":"2024-03-14T21:14:04.019342+0000","last_clean":"2024-03-14T21:14:04.019342+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T21:14:04.019342+0000","last_undegraded":"2024-03-14T21:14:04.019342+0000","last_fullsized":"2024-03-14T21:14:04.019342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:14.794102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017565+0000","last_change":"2024-03-14T21:14:02.653065+0000","last_active":"2024-03-14T21:14:04.017565+0000","last_peered":"2024-03-14T21:14:04.017565+0000","last_clean":"2024-03-14T21:14:04.017565+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T21:14:04.017565+0000","last_undegraded":"2024-03-14T21:14:04.017565+0000","last_fullsized":"2024-03-14T21:14:04.017565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.020019+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T21:14:04.020019+0000","last_peered":"2024-03-14T21:14:04.020019+0000","last_clean":"2024-03-14T21:14:04.020019+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T21:14:04.020019+0000","last_undegraded":"2024-03-14T21:14:04.020019+0000","last_fullsized":"2024-03-14T21:14:04.020019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954668+0000","last_change":"2024-03-14T21:14:02.659612+0000","last_active":"2024-03-14T21:14:04.954668+0000","last_peered":"2024-03-14T21:14:04.954668+0000","last_clean":"2024-03-14T21:14:04.954668+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T21:14:04.954668+0000","last_undegraded":"2024-03-14T21:14:04.954668+0000","last_fullsized":"2024-03-14T21:14:04.954668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017470+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T21:14:04.017470+0000","last_peered":"2024-03-14T21:14:04.017470+0000","last_clean":"2024-03-14T21:14:04.017470+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T21:14:04.017470+0000","last_undegraded":"2024-03-14T21:14:04.017470+0000","last_fullsized":"2024-03-14T21:14:04.017470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.927982+0000","last_change":"2024-03-14T21:14:02.662122+0000","last_active":"2024-03-14T21:14:04.927982+0000","last_peered":"2024-03-14T21:14:04.927982+0000","last_clean":"2024-03-14T21:14:04.927982+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T21:14:04.927982+0000","last_undegraded":"2024-03-14T21:14:04.927982+0000","last_fullsized":"2024-03-14T21:14:04.927982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.244002+0000","last_change":"2024-03-14T21:14:03.037921+0000","last_active":"2024-03-14T21:14:04.244002+0000","last_peered":"2024-03-14T21:14:04.244002+0000","last_clean":"2024-03-14T21:14:04.244002+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T21:14:04.244002+0000","last_undegraded":"2024-03-14T21:14:04.244002+0000","last_fullsized":"2024-03-14T21:14:04.244002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928891+0000","last_change":"2024-03-14T21:14:02.665745+0000","last_active":"2024-03-14T21:14:04.928891+0000","last_peered":"2024-03-14T21:14:04.928891+0000","last_clean":"2024-03-14T21:14:04.928891+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T21:14:04.928891+0000","last_undegraded":"2024-03-14T21:14:04.928891+0000","last_fullsized":"2024-03-14T21:14:04.928891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928718+0000","last_change":"2024-03-14T21:14:03.029269+0000","last_active":"2024-03-14T21:14:04.928718+0000","last_peered":"2024-03-14T21:14:04.928718+0000","last_clean":"2024-03-14T21:14:04.928718+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T21:14:04.928718+0000","last_undegraded":"2024-03-14T21:14:04.928718+0000","last_fullsized":"2024-03-14T21:14:04.928718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954436+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T21:14:04.954436+0000","last_peered":"2024-03-14T21:14:04.954436+0000","last_clean":"2024-03-14T21:14:04.954436+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T21:14:04.954436+0000","last_undegraded":"2024-03-14T21:14:04.954436+0000","last_fullsized":"2024-03-14T21:14:04.954436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019008+0000","last_change":"2024-03-14T21:14:03.039549+0000","last_active":"2024-03-14T21:14:04.019008+0000","last_peered":"2024-03-14T21:14:04.019008+0000","last_clean":"2024-03-14T21:14:04.019008+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T21:14:04.019008+0000","last_undegraded":"2024-03-14T21:14:04.019008+0000","last_fullsized":"2024-03-14T21:14:04.019008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:23.510516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017601+0000","last_change":"2024-03-14T21:14:02.660292+0000","last_active":"2024-03-14T21:14:04.017601+0000","last_peered":"2024-03-14T21:14:04.017601+0000","last_clean":"2024-03-14T21:14:04.017601+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T21:14:04.017601+0000","last_undegraded":"2024-03-14T21:14:04.017601+0000","last_fullsized":"2024-03-14T21:14:04.017601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243693+0000","last_change":"2024-03-14T21:14:03.040817+0000","last_active":"2024-03-14T21:14:04.243693+0000","last_peered":"2024-03-14T21:14:04.243693+0000","last_clean":"2024-03-14T21:14:04.243693+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T21:14:04.243693+0000","last_undegraded":"2024-03-14T21:14:04.243693+0000","last_fullsized":"2024-03-14T21:14:04.243693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.017553+0000","last_change":"2024-03-14T21:14:02.661073+0000","last_active":"2024-03-14T21:14:04.017553+0000","last_peered":"2024-03-14T21:14:04.017553+0000","last_clean":"2024-03-14T21:14:04.017553+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T21:14:04.017553+0000","last_undegraded":"2024-03-14T21:14:04.017553+0000","last_fullsized":"2024-03-14T21:14:04.017553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243996+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T21:14:04.243996+0000","last_peered":"2024-03-14T21:14:04.243996+0000","last_clean":"2024-03-14T21:14:04.243996+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T21:14:04.243996+0000","last_undegraded":"2024-03-14T21:14:04.243996+0000","last_fullsized":"2024-03-14T21:14:04.243996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954559+0000","last_change":"2024-03-14T21:14:02.663131+0000","last_active":"2024-03-14T21:14:04.954559+0000","last_peered":"2024-03-14T21:14:04.954559+0000","last_clean":"2024-03-14T21:14:04.954559+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T21:14:04.954559+0000","last_undegraded":"2024-03-14T21:14:04.954559+0000","last_fullsized":"2024-03-14T21:14:04.954559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928747+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T21:14:04.928747+0000","last_peered":"2024-03-14T21:14:04.928747+0000","last_clean":"2024-03-14T21:14:04.928747+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T21:14:04.928747+0000","last_undegraded":"2024-03-14T21:14:04.928747+0000","last_fullsized":"2024-03-14T21:14:04.928747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019966+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T21:14:04.019966+0000","last_peered":"2024-03-14T21:14:04.019966+0000","last_clean":"2024-03-14T21:14:04.019966+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T21:14:04.019966+0000","last_undegraded":"2024-03-14T21:14:04.019966+0000","last_fullsized":"2024-03-14T21:14:04.019966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954577+0000","last_change":"2024-03-14T21:14:02.660393+0000","last_active":"2024-03-14T21:14:04.954577+0000","last_peered":"2024-03-14T21:14:04.954577+0000","last_clean":"2024-03-14T21:14:04.954577+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T21:14:04.954577+0000","last_undegraded":"2024-03-14T21:14:04.954577+0000","last_fullsized":"2024-03-14T21:14:04.954577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.019982+0000","last_change":"2024-03-14T21:14:03.035149+0000","last_active":"2024-03-14T21:14:04.019982+0000","last_peered":"2024-03-14T21:14:04.019982+0000","last_clean":"2024-03-14T21:14:04.019982+0000","last_became_active":"2024-03-14T21:14:03.034649+0000","last_became_peered":"2024-03-14T21:14:03.034649+0000","last_unstale":"2024-03-14T21:14:04.019982+0000","last_undegraded":"2024-03-14T21:14:04.019982+0000","last_fullsized":"2024-03-14T21:14:04.019982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.954625+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T21:14:04.954625+0000","last_peered":"2024-03-14T21:14:04.954625+0000","last_clean":"2024-03-14T21:14:04.954625+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T21:14:04.954625+0000","last_undegraded":"2024-03-14T21:14:04.954625+0000","last_fullsized":"2024-03-14T21:14:04.954625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:48.203059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928726+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T21:14:04.928726+0000","last_peered":"2024-03-14T21:14:04.928726+0000","last_clean":"2024-03-14T21:14:04.928726+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T21:14:04.928726+0000","last_undegraded":"2024-03-14T21:14:04.928726+0000","last_fullsized":"2024-03-14T21:14:04.928726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243988+0000","last_change":"2024-03-14T21:14:02.654963+0000","last_active":"2024-03-14T21:14:04.243988+0000","last_peered":"2024-03-14T21:14:04.243988+0000","last_clean":"2024-03-14T21:14:04.243988+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T21:14:04.243988+0000","last_undegraded":"2024-03-14T21:14:04.243988+0000","last_fullsized":"2024-03-14T21:14:04.243988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.243950+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T21:14:04.243950+0000","last_peered":"2024-03-14T21:14:04.243950+0000","last_clean":"2024-03-14T21:14:04.243950+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T21:14:04.243950+0000","last_undegraded":"2024-03-14T21:14:04.243950+0000","last_fullsized":"2024-03-14T21:14:04.243950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T21:14:04.928743+0000","last_change":"2024-03-14T21:14:02.656125+0000","last_active":"2024-03-14T21:14:04.928743+0000","last_peered":"2024-03-14T21:14:04.928743+0000","last_clean":"2024-03-14T21:14:04.928743+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T21:14:04.928743+0000","last_undegraded":"2024-03-14T21:14:04.928743+0000","last_fullsized":"2024-03-14T21:14:04.928743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":819200,"data_stored":665956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T21:14:11.426 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-14T21:14:11.426 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-14T21:14:11.426 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-14T21:14:11.426 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.583+0000 7fd29ccd5640 1 Processor -- start 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.583+0000 7fd29ccd5640 1 -- start start 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd29ccd5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd29814b820 0x7fd29806a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd29ccd5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2980b3a20 con 0x7fd2980bb1c0 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd29ccd5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29814f090 con 0x7fd2980bad60 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd29ccd5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2980bb5f0 con 0x7fd29814b820 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd29659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd29814b820 0x7fd29806a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd29659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd29814b820 0x7fd29806a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43794/0 (socket says 172.21.15.49:43794) 2024-03-14T21:14:11.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd296d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd2980bb1c0 0x7fd29814e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46964/0 (socket says 172.21.15.49:46964) 2024-03-14T21:14:11.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.584+0000 7fd29659c640 1 -- 172.21.15.49:0/3781736333 learned_addr learned my addr 172.21.15.49:0/3781736333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:11.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.585+0000 7fd29659c640 1 -- 172.21.15.49:0/3781736333 >> v1:172.21.15.97:6789/0 conn(0x7fd2980bad60 legacy=0x7fd29814e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:11.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.585+0000 7fd29659c640 1 -- 172.21.15.49:0/3781736333 >> v1:172.21.15.49:6789/0 conn(0x7fd2980bb1c0 legacy=0x7fd29814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.585+0000 7fd29659c640 1 -- 172.21.15.49:0/3781736333 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980ba550 con 0x7fd29814b820 2024-03-14T21:14:11.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.585+0000 7fd29559a640 1 -- 172.21.15.49:0/3781736333 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202588648 0 0) 0x7fd2980b3a20 con 0x7fd2980bb1c0 2024-03-14T21:14:11.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.585+0000 7fd29659c640 1 --2- 172.21.15.49:0/3781736333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29814b820 0x7fd29806a6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd290009870 tx=0x7fd29002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f98ef6964dcfd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.585+0000 7fd29559a640 1 -- 172.21.15.49:0/3781736333 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290030860 con 0x7fd29814b820 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29559a640 1 -- 172.21.15.49:0/3781736333 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2900309c0 con 0x7fd29814b820 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29559a640 1 -- 172.21.15.49:0/3781736333 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290030d50 con 0x7fd29814b820 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29ccd5640 1 -- 172.21.15.49:0/3781736333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29814b820 msgr2=0x7fd29806a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/3781736333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29814b820 0x7fd29806a6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd290009870 tx=0x7fd29002b9a0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29ccd5640 1 -- 172.21.15.49:0/3781736333 shutdown_connections 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/3781736333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd29814b820 0x7fd29806a6e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29ccd5640 1 -- 172.21.15.49:0/3781736333 >> 172.21.15.49:0/3781736333 conn(0x7fd2980bcd80 msgr2=0x7fd2980bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29ccd5640 1 -- 172.21.15.49:0/3781736333 shutdown_connections 2024-03-14T21:14:11.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.586+0000 7fd29ccd5640 1 -- 172.21.15.49:0/3781736333 wait complete. 2024-03-14T21:14:11.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.587+0000 7fd29ccd5640 1 Processor -- start 2024-03-14T21:14:11.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.587+0000 7fd29ccd5640 1 -- start start 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29ccd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2980bad60 0x7fd2980a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29ccd5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd2980bb1c0 0x7fd2980b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29ccd5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29814b820 0x7fd298145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29ccd5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980a62a0 con 0x7fd2980bad60 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29ccd5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980b6110 con 0x7fd29814b820 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29ccd5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2981e3070 con 0x7fd2980bb1c0 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2980bad60 0x7fd2980a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2980bad60 0x7fd2980a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43808/0 (socket says 172.21.15.49:43808) 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 -- 172.21.15.49:0/40906636 learned_addr learned my addr 172.21.15.49:0/40906636 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:11.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd296d9d640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29814b820 0x7fd298145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd295d9b640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd2980bb1c0 0x7fd2980b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 -- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd2980bb1c0 msgr2=0x7fd2980b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd2980bb1c0 0x7fd2980b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 -- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29814b820 msgr2=0x7fd298145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29814b820 0x7fd298145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd29659c640 1 -- 172.21.15.49:0/40906636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900094d0 con 0x7fd2980bad60 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.588+0000 7fd296d9d640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29814b820 0x7fd298145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.589+0000 7fd29659c640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2980bad60 0x7fd2980a5d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd290009870 tx=0x7fd290004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.589+0000 7fd286ffd640 1 -- 172.21.15.49:0/40906636 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290039b60 con 0x7fd2980bad60 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.589+0000 7fd286ffd640 1 -- 172.21.15.49:0/40906636 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290039cc0 con 0x7fd2980bad60 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.589+0000 7fd286ffd640 1 -- 172.21.15.49:0/40906636 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290043630 con 0x7fd2980bad60 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.589+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2981e3350 con 0x7fd2980bad60 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.589+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981e3910 con 0x7fd2980bad60 2024-03-14T21:14:11.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.590+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298066bb0 con 0x7fd2980bad60 2024-03-14T21:14:11.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.590+0000 7fd286ffd640 1 -- 172.21.15.49:0/40906636 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd29002f3b0 con 0x7fd2980bad60 2024-03-14T21:14:11.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.591+0000 7fd286ffd640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd27806a7a0 0x7fd27806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.591+0000 7fd286ffd640 1 -- 172.21.15.49:0/40906636 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2900ab0e0 con 0x7fd2980bad60 2024-03-14T21:14:11.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.591+0000 7fd295d9b640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd27806a7a0 0x7fd27806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.592+0000 7fd295d9b640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd27806a7a0 0x7fd27806cc60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2980b3cc0 tx=0x7fd28c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.595+0000 7fd286ffd640 1 -- 172.21.15.49:0/40906636 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd290076500 con 0x7fd2980bad60 2024-03-14T21:14:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.811+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd29806b470 con 0x7fd2980bad60 2024-03-14T21:14:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.811+0000 7fd286ffd640 1 -- 172.21.15.49:0/40906636 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd290079b20 con 0x7fd2980bad60 2024-03-14T21:14:11.794 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:11.794 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-14T21:14:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd27806a7a0 msgr2=0x7fd27806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd27806a7a0 0x7fd27806cc60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2980b3cc0 tx=0x7fd28c009040 comp rx=0 tx=0).stop 2024-03-14T21:14:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2980bad60 msgr2=0x7fd2980a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2980bad60 0x7fd2980a5d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd290009870 tx=0x7fd290004730 comp rx=0 tx=0).stop 2024-03-14T21:14:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 shutdown_connections 2024-03-14T21:14:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd27806a7a0 0x7fd27806cc60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29814b820 0x7fd298145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd2980bb1c0 0x7fd2980b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 --2- 172.21.15.49:0/40906636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2980bad60 0x7fd2980a5d60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 >> 172.21.15.49:0/40906636 conn(0x7fd2980bcd80 msgr2=0x7fd29814aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 shutdown_connections 2024-03-14T21:14:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.813+0000 7fd29ccd5640 1 -- 172.21.15.49:0/40906636 wait complete. 2024-03-14T21:14:11.807 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-14T21:14:11.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:11.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.929+0000 7fe9b8f1b640 1 Processor -- start 2024-03-14T21:14:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.929+0000 7fe9b8f1b640 1 -- start start 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b8f1b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9b40b8c40 0x7fe9b414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b8f1b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b9080 0x7fe9b414e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b8f1b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b4071ad0 con 0x7fe9b4144eb0 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b8f1b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40be0f0 con 0x7fe9b40b8c40 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b8f1b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4070ec0 con 0x7fe9b40b9080 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b1d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9b40b8c40 0x7fe9b414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b1d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9b40b8c40 0x7fe9b414e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43818/0 (socket says 172.21.15.49:43818) 2024-03-14T21:14:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b1d9b640 1 -- 172.21.15.49:0/3051171478 learned_addr learned my addr 172.21.15.49:0/3051171478 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b2d9d640 1 --2- 172.21.15.49:0/3051171478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b9080 0x7fe9b414e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b1d9b640 1 -- 172.21.15.49:0/3051171478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b9080 msgr2=0x7fe9b414e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b1d9b640 1 --2- 172.21.15.49:0/3051171478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b9080 0x7fe9b414e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b1d9b640 1 -- 172.21.15.49:0/3051171478 >> v1:172.21.15.97:6789/0 conn(0x7fe9b4144eb0 legacy=0x7fe9b4070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.930+0000 7fe9b1d9b640 1 -- 172.21.15.49:0/3051171478 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40711a0 con 0x7fe9b40b8c40 2024-03-14T21:14:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.931+0000 7fe9b1d9b640 1 --2- 172.21.15.49:0/3051171478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b40b8c40 0x7fe9b414e0b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009870 tx=0x7fe99c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd008ca7ea28837 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.931+0000 7fe9b159a640 1 -- 172.21.15.49:0/3051171478 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c030840 con 0x7fe9b40b8c40 2024-03-14T21:14:11.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.931+0000 7fe9b159a640 1 -- 172.21.15.49:0/3051171478 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99c0309a0 con 0x7fe9b40b8c40 2024-03-14T21:14:11.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.931+0000 7fe9b159a640 1 -- 172.21.15.49:0/3051171478 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99c030d30 con 0x7fe9b40b8c40 2024-03-14T21:14:11.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/3051171478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b40b8c40 msgr2=0x7fe9b414e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/3051171478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b40b8c40 0x7fe9b414e0b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009870 tx=0x7fe99c02b980 comp rx=0 tx=0).stop 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/3051171478 shutdown_connections 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/3051171478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b9080 0x7fe9b414e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/3051171478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b40b8c40 0x7fe9b414e0b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/3051171478 >> 172.21.15.49:0/3051171478 conn(0x7fe9b40b9d30 msgr2=0x7fe9b40ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/3051171478 shutdown_connections 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.932+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/3051171478 wait complete. 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.933+0000 7fe9b8f1b640 1 Processor -- start 2024-03-14T21:14:11.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.933+0000 7fe9b8f1b640 1 -- start start 2024-03-14T21:14:11.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b8f1b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b8c40 0x7fe9b40a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b8f1b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40b9080 0x7fe9b41367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b8f1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4144eb0 0x7fe9b41e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b8f1b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40a2170 con 0x7fe9b4144eb0 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b8f1b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40bc810 con 0x7fe9b40b9080 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b8f1b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40bb310 con 0x7fe9b40b8c40 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4144eb0 0x7fe9b41e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4144eb0 0x7fe9b41e3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43834/0 (socket says 172.21.15.49:43834) 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 -- 172.21.15.49:0/896034292 learned_addr learned my addr 172.21.15.49:0/896034292 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b1d9b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40b9080 0x7fe9b41367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b259c640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b8c40 0x7fe9b40a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 -- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b8c40 msgr2=0x7fe9b40a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b8c40 0x7fe9b40a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 -- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40b9080 msgr2=0x7fe9b41367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:11.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40b9080 0x7fe9b41367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 -- 172.21.15.49:0/896034292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c0094d0 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b259c640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b8c40 0x7fe9b40a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b1d9b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40b9080 0x7fe9b41367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.934+0000 7fe9b2d9d640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4144eb0 0x7fe9b41e3ca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4007e40 tx=0x7fe9a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.935+0000 7fe9aaffd640 1 -- 172.21.15.49:0/896034292 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4014620 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.935+0000 7fe9aaffd640 1 -- 172.21.15.49:0/896034292 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a400ee90 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.935+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b40bc9a0 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.935+0000 7fe9aaffd640 1 -- 172.21.15.49:0/896034292 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40149e0 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.935+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b41353b0 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.936+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b40bcb30 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.936+0000 7fe9aaffd640 1 -- 172.21.15.49:0/896034292 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe9a4014b40 con 0x7fe9b4144eb0 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.936+0000 7fe9aaffd640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe99406a7a0 0x7fe99406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:11.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.937+0000 7fe9aaffd640 1 -- 172.21.15.49:0/896034292 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9a4089780 con 0x7fe9b4144eb0 2024-03-14T21:14:11.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.937+0000 7fe9b259c640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe99406a7a0 0x7fe99406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:11.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.937+0000 7fe9b259c640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe99406a7a0 0x7fe99406cc60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac007fd0 tx=0x7fe9ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:11.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:11.941+0000 7fe9aaffd640 1 -- 172.21.15.49:0/896034292 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe9a4054bb0 con 0x7fe9b4144eb0 2024-03-14T21:14:12.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.119+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9b40bc450 con 0x7fe9b4144eb0 2024-03-14T21:14:12.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.120+0000 7fe9aaffd640 1 -- 172.21.15.49:0/896034292 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fe9a40581d0 con 0x7fe9b4144eb0 2024-03-14T21:14:12.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:12.104 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:04.014218+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe99406a7a0 msgr2=0x7fe99406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe99406a7a0 0x7fe99406cc60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac007fd0 tx=0x7fe9ac009040 comp rx=0 tx=0).stop 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4144eb0 msgr2=0x7fe9b41e3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4144eb0 0x7fe9b41e3ca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4007e40 tx=0x7fe9a400e4b0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 shutdown_connections 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe99406a7a0 0x7fe99406cc60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9b4144eb0 0x7fe9b41e3ca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40b9080 0x7fe9b41367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 --2- 172.21.15.49:0/896034292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b40b8c40 0x7fe9b40a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 >> 172.21.15.49:0/896034292 conn(0x7fe9b40b9d30 msgr2=0x7fe9b4137c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 shutdown_connections 2024-03-14T21:14:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.122+0000 7fe9b8f1b640 1 -- 172.21.15.49:0/896034292 wait complete. 2024-03-14T21:14:12.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T21:13:50.964908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T21:14:01.020268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T21:14:01.995660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T21:14:12.117 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T21:14:12.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.242+0000 7fda5793c640 1 Processor -- start 2024-03-14T21:14:12.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.242+0000 7fda5793c640 1 -- start start 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda5793c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5006ba90 0x7fda500bbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda5793c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda500a5250 0x7fda500a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda5793c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda500bfbc0 con 0x7fda5006a950 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda5793c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda500bf670 con 0x7fda500a5250 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda5793c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda500bc950 con 0x7fda5006ba90 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda54ed7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5006ba90 0x7fda500bbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda55ed9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda500a5250 0x7fda500a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda556d8640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fda5006a950 0x7fda5014ac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:52110/0 (socket says 172.21.15.49:52110) 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda556d8640 1 -- 172.21.15.49:0/10683260 learned_addr learned my addr 172.21.15.49:0/10683260 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda54ed7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5006ba90 0x7fda500bbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:54006/0 (socket says 172.21.15.49:54006) 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda55ed9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda500a5250 0x7fda500a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:46220/0 (socket says 172.21.15.49:46220) 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda55ed9640 1 -- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5006ba90 msgr2=0x7fda500bbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:12.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda55ed9640 1 --2- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5006ba90 0x7fda500bbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.243+0000 7fda55ed9640 1 -- 172.21.15.49:0/10683260 >> v1:172.21.15.97:6789/0 conn(0x7fda5006a950 legacy=0x7fda5014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.244+0000 7fda4ffff640 1 -- 172.21.15.49:0/10683260 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155331952 0 0) 0x7fda500bfbc0 con 0x7fda5006a950 2024-03-14T21:14:12.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.244+0000 7fda55ed9640 1 -- 172.21.15.49:0/10683260 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500c0050 con 0x7fda500a5250 2024-03-14T21:14:12.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.244+0000 7fda54ed7640 1 --2- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5006ba90 0x7fda500bbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T21:14:12.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.244+0000 7fda55ed9640 1 --2- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 0x7fda500a56e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda40009d90 tx=0x7fda4002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8499d7398d9cf6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.244+0000 7fda4ffff640 1 -- 172.21.15.49:0/10683260 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4002cb30 con 0x7fda500a5250 2024-03-14T21:14:12.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda4ffff640 1 -- 172.21.15.49:0/10683260 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4002cc90 con 0x7fda500a5250 2024-03-14T21:14:12.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda4ffff640 1 -- 172.21.15.49:0/10683260 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40034440 con 0x7fda500a5250 2024-03-14T21:14:12.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 -- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 msgr2=0x7fda500a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 --2- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 0x7fda500a56e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda40009d90 tx=0x7fda4002bd80 comp rx=0 tx=0).stop 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 -- 172.21.15.49:0/10683260 shutdown_connections 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 --2- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 0x7fda500a56e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 --2- 172.21.15.49:0/10683260 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda5006ba90 0x7fda500bbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 -- 172.21.15.49:0/10683260 >> 172.21.15.49:0/10683260 conn(0x7fda500c0460 msgr2=0x7fda500c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 -- 172.21.15.49:0/10683260 shutdown_connections 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.245+0000 7fda5793c640 1 -- 172.21.15.49:0/10683260 wait complete. 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.246+0000 7fda5793c640 1 Processor -- start 2024-03-14T21:14:12.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.246+0000 7fda5793c640 1 -- start start 2024-03-14T21:14:12.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda5793c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda5006a950 0x7fda5014e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda5793c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 0x7fda500b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda5793c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda500b4d90 0x7fda50147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda5793c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5014eb80 con 0x7fda5006a950 2024-03-14T21:14:12.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda5793c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda500b3db0 con 0x7fda500a5250 2024-03-14T21:14:12.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda5793c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda500ba440 con 0x7fda500b4d90 2024-03-14T21:14:12.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda5006a950 0x7fda5014e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda5006a950 0x7fda5014e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43838/0 (socket says 172.21.15.49:43838) 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 -- 172.21.15.49:0/1247175564 learned_addr learned my addr 172.21.15.49:0/1247175564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda54ed7640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 0x7fda500b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda55ed9640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda500b4d90 0x7fda50147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 -- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda500b4d90 msgr2=0x7fda50147f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda500b4d90 0x7fda50147f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 -- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 msgr2=0x7fda500b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 0x7fda500b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 -- 172.21.15.49:0/1247175564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44009400 con 0x7fda5006a950 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.247+0000 7fda556d8640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda5006a950 0x7fda5014e640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda4800c460 tx=0x7fda4800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.248+0000 7fda4dffb640 1 -- 172.21.15.49:0/1247175564 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480074e0 con 0x7fda5006a950 2024-03-14T21:14:12.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.248+0000 7fda4dffb640 1 -- 172.21.15.49:0/1247175564 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda48007660 con 0x7fda5006a950 2024-03-14T21:14:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.248+0000 7fda4dffb640 1 -- 172.21.15.49:0/1247175564 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48004a20 con 0x7fda5006a950 2024-03-14T21:14:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.248+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda40009a70 con 0x7fda5006a950 2024-03-14T21:14:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.248+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda500b5840 con 0x7fda5006a950 2024-03-14T21:14:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.249+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5014acb0 con 0x7fda5006a950 2024-03-14T21:14:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.249+0000 7fda4dffb640 1 -- 172.21.15.49:0/1247175564 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fda48004b80 con 0x7fda5006a950 2024-03-14T21:14:12.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.250+0000 7fda4dffb640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda3c06a7d0 0x7fda3c06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.250+0000 7fda4dffb640 1 -- 172.21.15.49:0/1247175564 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda48087f20 con 0x7fda5006a950 2024-03-14T21:14:12.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.250+0000 7fda54ed7640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda3c06a7d0 0x7fda3c06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.250+0000 7fda54ed7640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda3c06a7d0 0x7fda3c06cc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda501dd770 tx=0x7fda44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.254+0000 7fda4dffb640 1 -- 172.21.15.49:0/1247175564 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fda480533d0 con 0x7fda5006a950 2024-03-14T21:14:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.431+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda500b5a30 con 0x7fda5006a950 2024-03-14T21:14:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.432+0000 7fda4dffb640 1 -- 172.21.15.49:0/1247175564 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda48018020 con 0x7fda5006a950 2024-03-14T21:14:12.415 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-14T21:14:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda3c06a7d0 msgr2=0x7fda3c06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda3c06a7d0 0x7fda3c06cc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda501dd770 tx=0x7fda44008040 comp rx=0 tx=0).stop 2024-03-14T21:14:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda5006a950 msgr2=0x7fda5014e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda5006a950 0x7fda5014e640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda4800c460 tx=0x7fda4800c920 comp rx=0 tx=0).stop 2024-03-14T21:14:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 shutdown_connections 2024-03-14T21:14:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fda3c06a7d0 0x7fda3c06cc90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fda500b4d90 0x7fda50147f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda500a5250 0x7fda500b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 --2- 172.21.15.49:0/1247175564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda5006a950 0x7fda5014e640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 >> 172.21.15.49:0/1247175564 conn(0x7fda500c0460 msgr2=0x7fda500bd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 shutdown_connections 2024-03-14T21:14:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.434+0000 7fda5793c640 1 -- 172.21.15.49:0/1247175564 wait complete. 2024-03-14T21:14:12.427 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T21:14:12.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.553+0000 7f8dc6c3e640 1 Processor -- start 2024-03-14T21:14:12.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.553+0000 7f8dc6c3e640 1 -- start start 2024-03-14T21:14:12.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc6c3e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8dc006ba30 0x7f8dc00bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc6c3e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc014b760 con 0x7f8dc00bebc0 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc6c3e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc014eca0 con 0x7f8dc00be7a0 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc6c3e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00be120 con 0x7f8dc006ba30 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc49da640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8dc006ba30 0x7f8dc00bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc51db640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8dc00bebc0 0x7f8dc014e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46966/0 (socket says 172.21.15.49:46966) 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc49da640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8dc006ba30 0x7f8dc00bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43840/0 (socket says 172.21.15.49:43840) 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc51db640 1 -- 172.21.15.49:0/3537147666 learned_addr learned my addr 172.21.15.49:0/3537147666 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc49da640 1 -- 172.21.15.49:0/3537147666 >> v1:172.21.15.97:6789/0 conn(0x7f8dc00be7a0 legacy=0x7f8dc014de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dbf7fe640 1 -- 172.21.15.49:0/3537147666 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647078588 0 0) 0x7f8dc014b760 con 0x7f8dc00bebc0 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc49da640 1 -- 172.21.15.49:0/3537147666 >> v1:172.21.15.49:6789/0 conn(0x7f8dc00bebc0 legacy=0x7f8dc014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.554+0000 7f8dc49da640 1 -- 172.21.15.49:0/3537147666 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc014a9a0 con 0x7f8dc006ba30 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.555+0000 7f8dc49da640 1 --2- 172.21.15.49:0/3537147666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc006ba30 0x7f8dc00bac10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8db40097a0 tx=0x7f8db402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9647ca142d61aaa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.555+0000 7f8dbf7fe640 1 -- 172.21.15.49:0/3537147666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db402f810 con 0x7f8dc006ba30 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.555+0000 7f8dbf7fe640 1 -- 172.21.15.49:0/3537147666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db402f970 con 0x7f8dc006ba30 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.555+0000 7f8dbf7fe640 1 -- 172.21.15.49:0/3537147666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db402fd00 con 0x7f8dc006ba30 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.556+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3537147666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc006ba30 msgr2=0x7f8dc00bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.556+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3537147666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc006ba30 0x7f8dc00bac10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8db40097a0 tx=0x7f8db402b9f0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.556+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3537147666 shutdown_connections 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.556+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3537147666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc006ba30 0x7f8dc00bac10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.556+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3537147666 >> 172.21.15.49:0/3537147666 conn(0x7f8dc00c0290 msgr2=0x7f8dc00c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.556+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3537147666 shutdown_connections 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.556+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3537147666 wait complete. 2024-03-14T21:14:12.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.557+0000 7f8dc6c3e640 1 Processor -- start 2024-03-14T21:14:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.557+0000 7f8dc6c3e640 1 -- start start 2024-03-14T21:14:12.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc006ba30 0x7f8dc00b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc00be7a0 0x7f8dc00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc00bebc0 0x7f8dc014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00b9800 con 0x7f8dc00be7a0 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00bf2a0 con 0x7f8dc00bebc0 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00a5450 con 0x7f8dc006ba30 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc00be7a0 0x7f8dc00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc00be7a0 0x7f8dc00b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43850/0 (socket says 172.21.15.49:43850) 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 -- 172.21.15.49:0/3726787209 learned_addr learned my addr 172.21.15.49:0/3726787209 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc51db640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc00bebc0 0x7f8dc014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc49da640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc006ba30 0x7f8dc00b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 -- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc006ba30 msgr2=0x7f8dc00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc006ba30 0x7f8dc00b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 -- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc00bebc0 msgr2=0x7f8dc014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc00bebc0 0x7f8dc014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 -- 172.21.15.49:0/3726787209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db4009400 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbffff640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc00be7a0 0x7f8dc00b8d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c3c0 tx=0x7f8dac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbd7fa640 1 -- 172.21.15.49:0/3726787209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0052d0 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbd7fa640 1 -- 172.21.15.49:0/3726787209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac005450 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc00a5730 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dbd7fa640 1 -- 172.21.15.49:0/3726787209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac003830 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.558+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00b6ea0 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.559+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc00a5c50 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.560+0000 7f8dbd7fa640 1 -- 172.21.15.49:0/3726787209 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8dac003990 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.560+0000 7f8dbd7fa640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8da406a7a0 0x7f8da406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.560+0000 7f8dbd7fa640 1 -- 172.21.15.49:0/3726787209 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dac0881e0 con 0x7f8dc00be7a0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.560+0000 7f8dc49da640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8da406a7a0 0x7f8da406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.561+0000 7f8dc49da640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8da406a7a0 0x7f8da406cc60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8db4036000 tx=0x7f8db4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.564+0000 7f8dbd7fa640 1 -- 172.21.15.49:0/3726787209 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8dac053690 con 0x7f8dc00be7a0 2024-03-14T21:14:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.742+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8dc0066bb0 con 0x7f8dc00be7a0 2024-03-14T21:14:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.742+0000 7f8dbd7fa640 1 -- 172.21.15.49:0/3726787209 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8dac056cb0 con 0x7f8dc00be7a0 2024-03-14T21:14:12.726 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8da406a7a0 msgr2=0x7f8da406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8da406a7a0 0x7f8da406cc60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8db4036000 tx=0x7f8db4008030 comp rx=0 tx=0).stop 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc00be7a0 msgr2=0x7f8dc00b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc00be7a0 0x7f8dc00b8d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c3c0 tx=0x7f8dac00c880 comp rx=0 tx=0).stop 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 shutdown_connections 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8da406a7a0 0x7f8da406cc60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dc00bebc0 0x7f8dc014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8dc00be7a0 0x7f8dc00b8d10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 --2- 172.21.15.49:0/3726787209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8dc006ba30 0x7f8dc00b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 >> 172.21.15.49:0/3726787209 conn(0x7f8dc00c0290 msgr2=0x7f8dc00a0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 shutdown_connections 2024-03-14T21:14:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.744+0000 7f8dc6c3e640 1 -- 172.21.15.49:0/3726787209 wait complete. 2024-03-14T21:14:12.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T21:14:12.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.863+0000 7fdd13036640 1 Processor -- start 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.863+0000 7fdd13036640 1 -- start start 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd13036640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 0x7fdd0c0bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd13036640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c0bb320 con 0x7fdd0c06ba30 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd13036640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c14b760 con 0x7fdd0c153200 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd13036640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0be120 con 0x7fdd0c0be7a0 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd10dd2640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdd0c06ba30 0x7fdd0c0bac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46980/0 (socket says 172.21.15.49:46980) 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd10dd2640 1 -- 172.21.15.49:0/3196717749 learned_addr learned my addr 172.21.15.49:0/3196717749 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807234793 0 0) 0x7fdd0c0bb320 con 0x7fdd0c06ba30 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf4003520 con 0x7fdd0c06ba30 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd03fff640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 0x7fdd0c0bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1765076956 0 0) 0x7fdcf4003520 con 0x7fdd0c06ba30 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 msgr2=0x7fdd0c0bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:12.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 0x7fdd0c0bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 >> v1:172.21.15.97:6789/0 conn(0x7fdd0c153200 legacy=0x7fdd0c1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c14a9a0 con 0x7fdd0c06ba30 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7fdd08001670 con 0x7fdd0c06ba30 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 >> v1:172.21.15.49:6789/0 conn(0x7fdd0c06ba30 legacy=0x7fdd0c0bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40039a0 0x7fdcf4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40047c0 0x7fdcf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 0x7fdcf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0be120 con 0x7fdcf40039a0 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf400bb40 con 0x7fdcf40047c0 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf400bd00 con 0x7fdd0c0be7a0 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.864+0000 7fdd10dd2640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40039a0 0x7fdcf4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd03fff640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40047c0 0x7fdcf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd115d3640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 0x7fdcf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd10dd2640 1 -- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 msgr2=0x7fdcf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd10dd2640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd0c0be7a0 0x7fdcf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd10dd2640 1 -- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40047c0 msgr2=0x7fdcf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd10dd2640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40047c0 0x7fdcf4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd10dd2640 1 -- 172.21.15.49:0/3196717749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4003520 con 0x7fdcf40039a0 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.865+0000 7fdd10dd2640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40039a0 0x7fdcf4003dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c240 tx=0x7fdcf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f863044c79477b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08005ad0 con 0x7fdcf40039a0 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd08005c50 con 0x7fdcf40039a0 2024-03-14T21:14:12.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd037fe640 1 -- 172.21.15.49:0/3196717749 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0802c8b0 con 0x7fdcf40039a0 2024-03-14T21:14:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd13036640 1 -- 172.21.15.49:0/3196717749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40039a0 msgr2=0x7fdcf4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd13036640 1 --2- 172.21.15.49:0/3196717749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40039a0 0x7fdcf4003dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c240 tx=0x7fdcf40057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd115d3640 1 -- 172.21.15.49:0/3196717749 reap_dead start 2024-03-14T21:14:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd13036640 1 -- 172.21.15.49:0/3196717749 shutdown_connections 2024-03-14T21:14:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd13036640 1 -- 172.21.15.49:0/3196717749 >> 172.21.15.49:0/3196717749 conn(0x7fdd0c0c0290 msgr2=0x7fdd0c0c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd13036640 1 -- 172.21.15.49:0/3196717749 shutdown_connections 2024-03-14T21:14:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.866+0000 7fdd13036640 1 -- 172.21.15.49:0/3196717749 wait complete. 2024-03-14T21:14:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.867+0000 7fdd13036640 1 Processor -- start 2024-03-14T21:14:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.867+0000 7fdd13036640 1 -- start start 2024-03-14T21:14:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd13036640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcf40039a0 0x7fdd0c0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd13036640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0c1418a0 0x7fdd0c1e3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd13036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c1e3660 0x7fdd0c13ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd13036640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c140250 con 0x7fdd0c1e3660 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd13036640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c1e3ae0 con 0x7fdd0c1418a0 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd13036640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0af510 con 0x7fdcf40039a0 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c1e3660 0x7fdd0c13ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c1e3660 0x7fdd0c13ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43870/0 (socket says 172.21.15.49:43870) 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 -- 172.21.15.49:0/1133809797 learned_addr learned my addr 172.21.15.49:0/1133809797 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd10dd2640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcf40039a0 0x7fdd0c0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd03fff640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0c1418a0 0x7fdd0c1e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 -- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcf40039a0 msgr2=0x7fdd0c0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcf40039a0 0x7fdd0c0aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 -- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0c1418a0 msgr2=0x7fdd0c1e3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0c1418a0 0x7fdd0c1e3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.868+0000 7fdd115d3640 1 -- 172.21.15.49:0/1133809797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c14a340 con 0x7fdd0c1e3660 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.869+0000 7fdd115d3640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c1e3660 0x7fdd0c13ed10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c14a8d0 tx=0x7fdd0400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.869+0000 7fdd017fa640 1 -- 172.21.15.49:0/1133809797 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003b80 con 0x7fdd0c1e3660 2024-03-14T21:14:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.869+0000 7fdd017fa640 1 -- 172.21.15.49:0/1133809797 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04003d00 con 0x7fdd0c1e3660 2024-03-14T21:14:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.869+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c0bf8d0 con 0x7fdd0c1e3660 2024-03-14T21:14:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.869+0000 7fdd017fa640 1 -- 172.21.15.49:0/1133809797 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04005310 con 0x7fdd0c1e3660 2024-03-14T21:14:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.869+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c149cf0 con 0x7fdd0c1e3660 2024-03-14T21:14:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.870+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c14a4d0 con 0x7fdd0c1e3660 2024-03-14T21:14:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.870+0000 7fdd017fa640 1 -- 172.21.15.49:0/1133809797 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fdd04005470 con 0x7fdd0c1e3660 2024-03-14T21:14:12.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.871+0000 7fdd017fa640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcf806a7a0 0x7fdcf806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:12.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.871+0000 7fdd017fa640 1 -- 172.21.15.49:0/1133809797 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd04088b70 con 0x7fdd0c1e3660 2024-03-14T21:14:12.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.871+0000 7fdd10dd2640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcf806a7a0 0x7fdcf806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:12.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.871+0000 7fdd10dd2640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcf806a7a0 0x7fdcf806cc60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd08034000 tx=0x7fdd08009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:12.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:12.875+0000 7fdd017fa640 1 -- 172.21.15.49:0/1133809797 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdd04054040 con 0x7fdd0c1e3660 2024-03-14T21:14:13.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.055+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd0c066bb0 con 0x7fdd0c1e3660 2024-03-14T21:14:13.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.055+0000 7fdd017fa640 1 -- 172.21.15.49:0/1133809797 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd04057660 con 0x7fdd0c1e3660 2024-03-14T21:14:13.039 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-14T21:14:13.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcf806a7a0 msgr2=0x7fdcf806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcf806a7a0 0x7fdcf806cc60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd08034000 tx=0x7fdd08009030 comp rx=0 tx=0).stop 2024-03-14T21:14:13.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c1e3660 msgr2=0x7fdd0c13ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c1e3660 0x7fdd0c13ed10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c14a8d0 tx=0x7fdd0400fd40 comp rx=0 tx=0).stop 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 shutdown_connections 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcf806a7a0 0x7fdcf806cc60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c1e3660 0x7fdd0c13ed10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0c1418a0 0x7fdd0c1e3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 --2- 172.21.15.49:0/1133809797 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcf40039a0 0x7fdd0c0aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 >> 172.21.15.49:0/1133809797 conn(0x7fdd0c0c0290 msgr2=0x7fdd0c09fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 shutdown_connections 2024-03-14T21:14:13.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.057+0000 7fdd13036640 1 -- 172.21.15.49:0/1133809797 wait complete. 2024-03-14T21:14:13.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T21:14:13.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.174+0000 7f74065d5640 1 Processor -- start 2024-03-14T21:14:13.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.174+0000 7f74065d5640 1 -- start start 2024-03-14T21:14:13.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.174+0000 7f74065d5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74000baef0 0x7f74001480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.174+0000 7f74065d5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74000bc940 0x7f7400155710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.174+0000 7f74065d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740006a6e0 con 0x7f740006b620 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.174+0000 7f74065d5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740006ad10 con 0x7f74000baef0 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.174+0000 7f74065d5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74000bc2b0 con 0x7f74000bc940 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f73fffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74000baef0 0x7f74001480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f73fffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74000baef0 0x7f74001480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43884/0 (socket says 172.21.15.49:43884) 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f73fffff640 1 -- 172.21.15.49:0/2769065011 learned_addr learned my addr 172.21.15.49:0/2769065011 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f7404b72640 1 --2- 172.21.15.49:0/2769065011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74000bc940 0x7f7400155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f73fffff640 1 -- 172.21.15.49:0/2769065011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74000bc940 msgr2=0x7f7400155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f73fffff640 1 --2- 172.21.15.49:0/2769065011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74000bc940 0x7f7400155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f73fffff640 1 -- 172.21.15.49:0/2769065011 >> v1:172.21.15.97:6789/0 conn(0x7f740006b620 legacy=0x7f74000bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f73fffff640 1 -- 172.21.15.49:0/2769065011 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740006bab0 con 0x7f74000baef0 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.175+0000 7f7404b72640 1 --2- 172.21.15.49:0/2769065011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74000bc940 0x7f7400155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f73fffff640 1 --2- 172.21.15.49:0/2769065011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000baef0 0x7f74001480b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009870 tx=0x7f73f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8c0d85b569e7e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f73feffd640 1 -- 172.21.15.49:0/2769065011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0031640 con 0x7f74000baef0 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f73feffd640 1 -- 172.21.15.49:0/2769065011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f00317a0 con 0x7f74000baef0 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f73feffd640 1 -- 172.21.15.49:0/2769065011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0031b30 con 0x7f74000baef0 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 -- 172.21.15.49:0/2769065011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000baef0 msgr2=0x7f74001480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 --2- 172.21.15.49:0/2769065011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000baef0 0x7f74001480b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009870 tx=0x7f73f002b8f0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 -- 172.21.15.49:0/2769065011 shutdown_connections 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 --2- 172.21.15.49:0/2769065011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74000bc940 0x7f7400155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 --2- 172.21.15.49:0/2769065011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000baef0 0x7f74001480b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 -- 172.21.15.49:0/2769065011 >> 172.21.15.49:0/2769065011 conn(0x7f74000bcd80 msgr2=0x7f74000bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 -- 172.21.15.49:0/2769065011 shutdown_connections 2024-03-14T21:14:13.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.176+0000 7f74065d5640 1 -- 172.21.15.49:0/2769065011 wait complete. 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.177+0000 7f74065d5640 1 Processor -- start 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f74065d5640 1 -- start start 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f74065d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74000baef0 0x7f740014c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f74065d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000bc940 0x7f74000b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f74065d5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f740014cee0 0x7f740014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f74065d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740014db50 con 0x7f74000bc940 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f74065d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740009fa60 con 0x7f74000baef0 2024-03-14T21:14:13.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f74065d5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74001e0940 con 0x7f740014cee0 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f73ff7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000bc940 0x7f74000b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.178+0000 7f73ff7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000bc940 0x7f74000b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43892/0 (socket says 172.21.15.49:43892) 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73ff7fe640 1 -- 172.21.15.49:0/3229793266 learned_addr learned my addr 172.21.15.49:0/3229793266 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74000baef0 0x7f740014c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f7404b72640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f740014cee0 0x7f740014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73ff7fe640 1 -- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f740014cee0 msgr2=0x7f740014e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73ff7fe640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f740014cee0 0x7f740014e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73ff7fe640 1 -- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74000baef0 msgr2=0x7f740014c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73ff7fe640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74000baef0 0x7f740014c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73ff7fe640 1 -- 172.21.15.49:0/3229793266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f00094d0 con 0x7f74000bc940 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73ff7fe640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000bc940 0x7f74000b2360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c450 tx=0x7f73f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73fcff9640 1 -- 172.21.15.49:0/3229793266 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f40052d0 con 0x7f74000bc940 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73fcff9640 1 -- 172.21.15.49:0/3229793266 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f4005430 con 0x7f74000bc940 2024-03-14T21:14:13.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74001dfb80 con 0x7f74000bc940 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f73fcff9640 1 -- 172.21.15.49:0/3229793266 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4012870 con 0x7f74000bc940 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.179+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74001e0140 con 0x7f74000bc940 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.180+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7400066bb0 con 0x7f74000bc940 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.181+0000 7f73fcff9640 1 -- 172.21.15.49:0/3229793266 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f73f40129d0 con 0x7f74000bc940 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.181+0000 7f73fcff9640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f73ec06a7a0 0x7f73ec06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.181+0000 7f73fcff9640 1 -- 172.21.15.49:0/3229793266 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73f4088720 con 0x7f74000bc940 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.181+0000 7f73fffff640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f73ec06a7a0 0x7f73ec06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.182+0000 7f73fffff640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f73ec06a7a0 0x7f73ec06cc60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73f0036000 tx=0x7f73f003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.185+0000 7f73fcff9640 1 -- 172.21.15.49:0/3229793266 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f73f4053bf0 con 0x7f74000bc940 2024-03-14T21:14:13.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.401+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74000b3a20 con 0x7f74000bc940 2024-03-14T21:14:13.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.402+0000 7f73fcff9640 1 -- 172.21.15.49:0/3229793266 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3969 (secure 0 0 0) 0x7f73f4057210 con 0x7f74000bc940 2024-03-14T21:14:13.385 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:13.385 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2343109927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2343109927},{"type":"v1","addr":"172.21.15.97:6835","nonce":2343109927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1491968682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1491968682},{"type":"v1","addr":"172.21.15.49:6833","nonce":1491968682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T21:14:03.012761+0000","modified":"2024-03-14T21:14:05.655989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T21:14:13.386 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f73ec06a7a0 msgr2=0x7f73ec06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f73ec06a7a0 0x7f73ec06cc60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73f0036000 tx=0x7f73f003c040 comp rx=0 tx=0).stop 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000bc940 msgr2=0x7f74000b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000bc940 0x7f74000b2360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c450 tx=0x7f73f400c910 comp rx=0 tx=0).stop 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 shutdown_connections 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f73ec06a7a0 0x7f73ec06cc60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f740014cee0 0x7f740014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74000bc940 0x7f74000b2360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 --2- 172.21.15.49:0/3229793266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74000baef0 0x7f740014c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 >> 172.21.15.49:0/3229793266 conn(0x7f74000bcd80 msgr2=0x7f740014a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:13.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 shutdown_connections 2024-03-14T21:14:13.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.404+0000 7f74065d5640 1 -- 172.21.15.49:0/3229793266 wait complete. 2024-03-14T21:14:13.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:13.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.520+0000 7fe21db0a640 1 Processor -- start 2024-03-14T21:14:13.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.520+0000 7fe21db0a640 1 -- start start 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe21db0a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2180baef0 0x7fe2181480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe21db0a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe2180bc940 0x7fe218155710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe21db0a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21806a6e0 con 0x7fe21806b620 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe21db0a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21806ad10 con 0x7fe2180baef0 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe21db0a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2180bc2b0 con 0x7fe2180bc940 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe2177fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2180baef0 0x7fe2181480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe2177fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2180baef0 0x7fe2181480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43908/0 (socket says 172.21.15.49:43908) 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe2177fe640 1 -- 172.21.15.49:0/1359989420 learned_addr learned my addr 172.21.15.49:0/1359989420 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe217fff640 1 --2- 172.21.15.49:0/1359989420 >> v2:172.21.15.97:3300/0 conn(0x7fe2180bc940 0x7fe218155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe2177fe640 1 -- 172.21.15.49:0/1359989420 >> v2:172.21.15.97:3300/0 conn(0x7fe2180bc940 msgr2=0x7fe218155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe2177fe640 1 --2- 172.21.15.49:0/1359989420 >> v2:172.21.15.97:3300/0 conn(0x7fe2180bc940 0x7fe218155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe2177fe640 1 -- 172.21.15.49:0/1359989420 >> v1:172.21.15.97:6789/0 conn(0x7fe21806b620 legacy=0x7fe2180bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.521+0000 7fe2177fe640 1 -- 172.21.15.49:0/1359989420 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21806bab0 con 0x7fe2180baef0 2024-03-14T21:14:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.522+0000 7fe2177fe640 1 --2- 172.21.15.49:0/1359989420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe2181480b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe21814f480 tx=0x7fe20802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b73da0a983fe34 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.522+0000 7fe2167fc640 1 -- 172.21.15.49:0/1359989420 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2080317e0 con 0x7fe2180baef0 2024-03-14T21:14:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.522+0000 7fe2167fc640 1 -- 172.21.15.49:0/1359989420 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208031940 con 0x7fe2180baef0 2024-03-14T21:14:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.522+0000 7fe2167fc640 1 -- 172.21.15.49:0/1359989420 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208031cd0 con 0x7fe2180baef0 2024-03-14T21:14:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.522+0000 7fe21db0a640 1 -- 172.21.15.49:0/1359989420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 msgr2=0x7fe2181480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.522+0000 7fe21db0a640 1 --2- 172.21.15.49:0/1359989420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe2181480b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe21814f480 tx=0x7fe20802b880 comp rx=0 tx=0).stop 2024-03-14T21:14:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.523+0000 7fe21db0a640 1 -- 172.21.15.49:0/1359989420 shutdown_connections 2024-03-14T21:14:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.523+0000 7fe21db0a640 1 --2- 172.21.15.49:0/1359989420 >> v2:172.21.15.97:3300/0 conn(0x7fe2180bc940 0x7fe218155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.523+0000 7fe21db0a640 1 --2- 172.21.15.49:0/1359989420 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe2181480b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.523+0000 7fe21db0a640 1 -- 172.21.15.49:0/1359989420 >> 172.21.15.49:0/1359989420 conn(0x7fe2180bcd80 msgr2=0x7fe2180bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.523+0000 7fe21db0a640 1 -- 172.21.15.49:0/1359989420 shutdown_connections 2024-03-14T21:14:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.523+0000 7fe21db0a640 1 -- 172.21.15.49:0/1359989420 wait complete. 2024-03-14T21:14:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.524+0000 7fe21db0a640 1 Processor -- start 2024-03-14T21:14:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.524+0000 7fe21db0a640 1 -- start start 2024-03-14T21:14:13.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe21db0a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe21814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe21db0a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2180bc940 0x7fe2180a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe21db0a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2180b69e0 0x7fe2181e0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe21db0a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180a2b40 con 0x7fe2180baef0 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe21db0a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21814edd0 con 0x7fe2180bc940 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe21db0a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21814c070 con 0x7fe2180b69e0 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe21814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe21814dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43910/0 (socket says 172.21.15.49:43910) 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe216ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2180bc940 0x7fe2180a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 -- 172.21.15.49:0/3446446285 learned_addr learned my addr 172.21.15.49:0/3446446285 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe217fff640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2180b69e0 0x7fe2181e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 -- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2180b69e0 msgr2=0x7fe2181e0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2180b69e0 0x7fe2181e0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 -- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2180bc940 msgr2=0x7fe2180a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2180bc940 0x7fe2180a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.525+0000 7fe2177fe640 1 -- 172.21.15.49:0/3446446285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080094d0 con 0x7fe2180baef0 2024-03-14T21:14:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.526+0000 7fe2177fe640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe21814dbf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe20802b8c0 tx=0x7fe2080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.527+0000 7fe1fffff640 1 -- 172.21.15.49:0/3446446285 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208032880 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.527+0000 7fe1fffff640 1 -- 172.21.15.49:0/3446446285 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208032a20 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.527+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21814c2e0 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.527+0000 7fe1fffff640 1 -- 172.21.15.49:0/3446446285 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208032dd0 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.527+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21814c8a0 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.528+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21814ca90 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.528+0000 7fe1fffff640 1 -- 172.21.15.49:0/3446446285 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe2080453f0 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.528+0000 7fe1fffff640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe20406a7a0 0x7fe20406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.529+0000 7fe1fffff640 1 -- 172.21.15.49:0/3446446285 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2080ab110 con 0x7fe2180baef0 2024-03-14T21:14:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.529+0000 7fe216ffd640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe20406a7a0 0x7fe20406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.529+0000 7fe216ffd640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe20406a7a0 0x7fe20406cc60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2181dff80 tx=0x7fe20c007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.533+0000 7fe1fffff640 1 -- 172.21.15.49:0/3446446285 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe208076850 con 0x7fe2180baef0 2024-03-14T21:14:13.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.716+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2180666a0 con 0x7fe2180baef0 2024-03-14T21:14:13.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.717+0000 7fe1fffff640 1 -- 172.21.15.49:0/3446446285 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fe208079e70 con 0x7fe2180baef0 2024-03-14T21:14:13.701 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:13.701 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:04.014218+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:13.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe20406a7a0 msgr2=0x7fe20406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe20406a7a0 0x7fe20406cc60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2181dff80 tx=0x7fe20c007e30 comp rx=0 tx=0).stop 2024-03-14T21:14:13.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 msgr2=0x7fe21814dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe21814dbf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe20802b8c0 tx=0x7fe2080046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 shutdown_connections 2024-03-14T21:14:13.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe20406a7a0 0x7fe20406cc60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2180b69e0 0x7fe2181e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2180bc940 0x7fe2180a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 --2- 172.21.15.49:0/3446446285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2180baef0 0x7fe21814dbf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 >> 172.21.15.49:0/3446446285 conn(0x7fe2180bcd80 msgr2=0x7fe218072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:13.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 shutdown_connections 2024-03-14T21:14:13.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.720+0000 7fe21db0a640 1 -- 172.21.15.49:0/3446446285 wait complete. 2024-03-14T21:14:13.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T21:13:50.964908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T21:14:01.020268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T21:14:01.995660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T21:14:13.714 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T21:14:13.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.839+0000 7f8b66a05640 1 Processor -- start 2024-03-14T21:14:13.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b66a05640 1 -- start start 2024-03-14T21:14:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b66a05640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6006a950 0x7f8b600a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b66a05640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b60153580 0x7f8b60155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b66a05640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b600bac00 con 0x7f8b600bc850 2024-03-14T21:14:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b66a05640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6006b800 con 0x7f8b60153580 2024-03-14T21:14:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b66a05640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6006bae0 con 0x7f8b6006a950 2024-03-14T21:14:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b64fa2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b60153580 0x7f8b60155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b64fa2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b60153580 0x7f8b60155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43926/0 (socket says 172.21.15.49:43926) 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b64fa2640 1 -- 172.21.15.49:0/2214963653 learned_addr learned my addr 172.21.15.49:0/2214963653 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b5f7fe640 1 --1- 172.21.15.49:0/2214963653 >> v1:172.21.15.49:6789/0 conn(0x7f8b600bc850 0x7f8b6014ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46996/0 (socket says 172.21.15.49:46996) 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.840+0000 7f8b5ffff640 1 --2- 172.21.15.49:0/2214963653 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6006a950 0x7f8b600a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.841+0000 7f8b64fa2640 1 -- 172.21.15.49:0/2214963653 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6006a950 msgr2=0x7f8b600a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.841+0000 7f8b64fa2640 1 --2- 172.21.15.49:0/2214963653 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6006a950 0x7f8b600a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.841+0000 7f8b5effd640 1 -- 172.21.15.49:0/2214963653 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897473148 0 0) 0x7f8b600bac00 con 0x7f8b600bc850 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.841+0000 7f8b64fa2640 1 -- 172.21.15.49:0/2214963653 >> v1:172.21.15.49:6789/0 conn(0x7f8b600bc850 legacy=0x7f8b6014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.841+0000 7f8b64fa2640 1 -- 172.21.15.49:0/2214963653 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b600bae50 con 0x7f8b60153580 2024-03-14T21:14:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.841+0000 7f8b64fa2640 1 --2- 172.21.15.49:0/2214963653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b60153580 0x7f8b60155970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009e50 tx=0x7f8b5802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cadb1abcc20cf33 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b5effd640 1 -- 172.21.15.49:0/2214963653 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5802cb00 con 0x7f8b60153580 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b5effd640 1 -- 172.21.15.49:0/2214963653 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5802cc60 con 0x7f8b60153580 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b5effd640 1 -- 172.21.15.49:0/2214963653 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580343f0 con 0x7f8b60153580 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 -- 172.21.15.49:0/2214963653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b60153580 msgr2=0x7f8b60155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2214963653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b60153580 0x7f8b60155970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009e50 tx=0x7f8b5802bf60 comp rx=0 tx=0).stop 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 -- 172.21.15.49:0/2214963653 shutdown_connections 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2214963653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b60153580 0x7f8b60155970 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2214963653 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6006a950 0x7f8b600a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 -- 172.21.15.49:0/2214963653 >> 172.21.15.49:0/2214963653 conn(0x7f8b600c0280 msgr2=0x7f8b600c0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 -- 172.21.15.49:0/2214963653 shutdown_connections 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.842+0000 7f8b66a05640 1 -- 172.21.15.49:0/2214963653 wait complete. 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.843+0000 7f8b66a05640 1 Processor -- start 2024-03-14T21:14:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b66a05640 1 -- start start 2024-03-14T21:14:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b66a05640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b6006a950 0x7f8b600a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b66a05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b600bc850 0x7f8b600b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b66a05640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b60153580 0x7f8b601e3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b66a05640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600ba470 con 0x7f8b6006a950 2024-03-14T21:14:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b66a05640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600a0550 con 0x7f8b600bc850 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b66a05640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600b3db0 con 0x7f8b60153580 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b6006a950 0x7f8b600a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b6006a950 0x7f8b600a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43940/0 (socket says 172.21.15.49:43940) 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b5ffff640 1 -- 172.21.15.49:0/2880361722 learned_addr learned my addr 172.21.15.49:0/2880361722 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b64fa2640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b60153580 0x7f8b601e3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.844+0000 7f8b5f7fe640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b600bc850 0x7f8b600b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5ffff640 1 -- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b60153580 msgr2=0x7f8b601e3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5ffff640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b60153580 0x7f8b601e3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5ffff640 1 -- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b600bc850 msgr2=0x7f8b600b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5ffff640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b600bc850 0x7f8b600b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5ffff640 1 -- 172.21.15.49:0/2880361722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b58009b30 con 0x7f8b6006a950 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5ffff640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b6006a950 0x7f8b600a0010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000e8a0 tx=0x7f8b5000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5cff9640 1 -- 172.21.15.49:0/2880361722 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50009510 con 0x7f8b6006a950 2024-03-14T21:14:13.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5cff9640 1 -- 172.21.15.49:0/2880361722 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5000c940 con 0x7f8b6006a950 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b600b50c0 con 0x7f8b6006a950 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b5cff9640 1 -- 172.21.15.49:0/2880361722 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000ccf0 con 0x7f8b6006a950 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.845+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b600ba090 con 0x7f8b6006a950 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.846+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b600ba5b0 con 0x7f8b6006a950 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.846+0000 7f8b5cff9640 1 -- 172.21.15.49:0/2880361722 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8b50013470 con 0x7f8b6006a950 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.847+0000 7f8b5cff9640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8b4c06a7a0 0x7f8b4c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.847+0000 7f8b5cff9640 1 -- 172.21.15.49:0/2880361722 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b50088e00 con 0x7f8b6006a950 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.847+0000 7f8b5f7fe640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8b4c06a7a0 0x7f8b4c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.848+0000 7f8b5f7fe640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8b4c06a7a0 0x7f8b4c06cc60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b60145ec0 tx=0x7f8b54007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:13.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:13.851+0000 7f8b5cff9640 1 -- 172.21.15.49:0/2880361722 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8b600ba5b0 con 0x7f8b6006a950 2024-03-14T21:14:14.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.030+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b600ba5b0 con 0x7f8b6006a950 2024-03-14T21:14:14.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.030+0000 7f8b5cff9640 1 -- 172.21.15.49:0/2880361722 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b600ba5b0 con 0x7f8b6006a950 2024-03-14T21:14:14.014 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-14T21:14:14.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8b4c06a7a0 msgr2=0x7f8b4c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8b4c06a7a0 0x7f8b4c06cc60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b60145ec0 tx=0x7f8b54007d50 comp rx=0 tx=0).stop 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b6006a950 msgr2=0x7f8b600a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b6006a950 0x7f8b600a0010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000e8a0 tx=0x7f8b5000eca0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 shutdown_connections 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8b4c06a7a0 0x7f8b4c06cc60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b60153580 0x7f8b601e3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b600bc850 0x7f8b600b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 --2- 172.21.15.49:0/2880361722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b6006a950 0x7f8b600a0010 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 >> 172.21.15.49:0/2880361722 conn(0x7f8b600c0280 msgr2=0x7f8b600a3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 shutdown_connections 2024-03-14T21:14:14.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.032+0000 7f8b66a05640 1 -- 172.21.15.49:0/2880361722 wait complete. 2024-03-14T21:14:14.026 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T21:14:14.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.151+0000 7f62965d2640 1 Processor -- start 2024-03-14T21:14:14.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.151+0000 7f62965d2640 1 -- start start 2024-03-14T21:14:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f62965d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f629006ba30 0x7f62900bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f62965d2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62900be7a0 0x7f62900bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f62965d2640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629014b760 con 0x7f629014de80 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f62965d2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900be120 con 0x7f62900be7a0 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f62965d2640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629014a9a0 con 0x7f629006ba30 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62900be7a0 0x7f62900bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62900be7a0 0x7f62900bebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43946/0 (socket says 172.21.15.49:43946) 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628f7fe640 1 -- 172.21.15.49:0/2267458511 learned_addr learned my addr 172.21.15.49:0/2267458511 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628ffff640 1 --2- 172.21.15.49:0/2267458511 >> v2:172.21.15.97:3300/0 conn(0x7f629006ba30 0x7f62900bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628f7fe640 1 -- 172.21.15.49:0/2267458511 >> v2:172.21.15.97:3300/0 conn(0x7f629006ba30 msgr2=0x7f62900bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628f7fe640 1 --2- 172.21.15.49:0/2267458511 >> v2:172.21.15.97:3300/0 conn(0x7f629006ba30 0x7f62900bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628f7fe640 1 -- 172.21.15.49:0/2267458511 >> v1:172.21.15.97:6789/0 conn(0x7f629014de80 legacy=0x7f629014e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.152+0000 7f628f7fe640 1 -- 172.21.15.49:0/2267458511 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629014ac80 con 0x7f62900be7a0 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.153+0000 7f628f7fe640 1 --2- 172.21.15.49:0/2267458511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900bebc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62900bc080 tx=0x7f628002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5537226bd9483f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.153+0000 7f628effd640 1 -- 172.21.15.49:0/2267458511 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280030860 con 0x7f62900be7a0 2024-03-14T21:14:14.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.153+0000 7f628effd640 1 -- 172.21.15.49:0/2267458511 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62800309c0 con 0x7f62900be7a0 2024-03-14T21:14:14.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.153+0000 7f628effd640 1 -- 172.21.15.49:0/2267458511 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280030d50 con 0x7f62900be7a0 2024-03-14T21:14:14.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 -- 172.21.15.49:0/2267458511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 msgr2=0x7f62900bebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 --2- 172.21.15.49:0/2267458511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900bebc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62900bc080 tx=0x7f628002b9a0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 -- 172.21.15.49:0/2267458511 shutdown_connections 2024-03-14T21:14:14.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 --2- 172.21.15.49:0/2267458511 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900bebc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 --2- 172.21.15.49:0/2267458511 >> v2:172.21.15.97:3300/0 conn(0x7f629006ba30 0x7f62900bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 -- 172.21.15.49:0/2267458511 >> 172.21.15.49:0/2267458511 conn(0x7f62900c0290 msgr2=0x7f62900c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:14.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 -- 172.21.15.49:0/2267458511 shutdown_connections 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.154+0000 7f62965d2640 1 -- 172.21.15.49:0/2267458511 wait complete. 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.155+0000 7f62965d2640 1 Processor -- start 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.155+0000 7f62965d2640 1 -- start start 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f62965d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f629006ba30 0x7f62901d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f62965d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f62965d2640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f629014de80 0x7f6290143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f62965d2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629009f720 con 0x7f62900be7a0 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f62965d2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62901d5fe0 con 0x7f629006ba30 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f62965d2640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62901d6b60 con 0x7f629014de80 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43954/0 (socket says 172.21.15.49:43954) 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 -- 172.21.15.49:0/3175673136 learned_addr learned my addr 172.21.15.49:0/3175673136 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628ffff640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f629006ba30 0x7f62901d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f6294b6f640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f629014de80 0x7f6290143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 -- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f629014de80 msgr2=0x7f6290143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f629014de80 0x7f6290143410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 -- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f629006ba30 msgr2=0x7f62901d5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f629006ba30 0x7f62901d5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 -- 172.21.15.49:0/3175673136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62800094d0 con 0x7f62900be7a0 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f6294b6f640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f629014de80 0x7f6290143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628f7fe640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900b8a60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6280000c00 tx=0x7f6280004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.156+0000 7f628cff9640 1 -- 172.21.15.49:0/3175673136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280039c90 con 0x7f62900be7a0 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.157+0000 7f628cff9640 1 -- 172.21.15.49:0/3175673136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6280039df0 con 0x7f62900be7a0 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.157+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62900b8640 con 0x7f62900be7a0 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.157+0000 7f628cff9640 1 -- 172.21.15.49:0/3175673136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280043620 con 0x7f62900be7a0 2024-03-14T21:14:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.157+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62900a5800 con 0x7f62900be7a0 2024-03-14T21:14:14.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.158+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629009f9a0 con 0x7f62900be7a0 2024-03-14T21:14:14.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.158+0000 7f628cff9640 1 -- 172.21.15.49:0/3175673136 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6280043780 con 0x7f62900be7a0 2024-03-14T21:14:14.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.158+0000 7f628cff9640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f627c06a7a0 0x7f627c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.158+0000 7f628cff9640 1 -- 172.21.15.49:0/3175673136 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f628003e070 con 0x7f62900be7a0 2024-03-14T21:14:14.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.159+0000 7f628ffff640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f627c06a7a0 0x7f627c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.159+0000 7f628ffff640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f627c06a7a0 0x7f627c06cc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f62840079a0 tx=0x7f6284008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.163+0000 7f628cff9640 1 -- 172.21.15.49:0/3175673136 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f629009f9a0 con 0x7f62900be7a0 2024-03-14T21:14:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.341+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62900a5380 con 0x7f62900be7a0 2024-03-14T21:14:14.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.342+0000 7f628cff9640 1 -- 172.21.15.49:0/3175673136 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62900a5380 con 0x7f62900be7a0 2024-03-14T21:14:14.325 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-14T21:14:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f627c06a7a0 msgr2=0x7f627c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f627c06a7a0 0x7f627c06cc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f62840079a0 tx=0x7f6284008040 comp rx=0 tx=0).stop 2024-03-14T21:14:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 msgr2=0x7f62900b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900b8a60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6280000c00 tx=0x7f6280004730 comp rx=0 tx=0).stop 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 shutdown_connections 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f627c06a7a0 0x7f627c06cc60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f629014de80 0x7f6290143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62900be7a0 0x7f62900b8a60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 --2- 172.21.15.49:0/3175673136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f629006ba30 0x7f62901d5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 >> 172.21.15.49:0/3175673136 conn(0x7f62900c0290 msgr2=0x7f62900a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 shutdown_connections 2024-03-14T21:14:14.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.344+0000 7f62965d2640 1 -- 172.21.15.49:0/3175673136 wait complete. 2024-03-14T21:14:14.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T21:14:14.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe530d63640 1 Processor -- start 2024-03-14T21:14:14.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe530d63640 1 -- start start 2024-03-14T21:14:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe530d63640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe52c14bde0 0x7fe52c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe530d63640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52c14ccc0 con 0x7fe52c148bc0 2024-03-14T21:14:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe530d63640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52c1462a0 con 0x7fe52c071b10 2024-03-14T21:14:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe530d63640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52c0b3ec0 con 0x7fe52c14bde0 2024-03-14T21:14:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe523fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe52c14bde0 0x7fe52c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe523fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe52c14bde0 0x7fe52c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43956/0 (socket says 172.21.15.49:43956) 2024-03-14T21:14:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.463+0000 7fe52ad9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe52c148bc0 0x7fe52c149030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47004/0 (socket says 172.21.15.49:47004) 2024-03-14T21:14:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.464+0000 7fe523fff640 1 -- 172.21.15.49:0/1934752134 learned_addr learned my addr 172.21.15.49:0/1934752134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.464+0000 7fe523fff640 1 -- 172.21.15.49:0/1934752134 >> v1:172.21.15.97:6789/0 conn(0x7fe52c071b10 legacy=0x7fe52c145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.464+0000 7fe523fff640 1 -- 172.21.15.49:0/1934752134 >> v1:172.21.15.49:6789/0 conn(0x7fe52c148bc0 legacy=0x7fe52c149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.464+0000 7fe523fff640 1 -- 172.21.15.49:0/1934752134 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c14c390 con 0x7fe52c14bde0 2024-03-14T21:14:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.464+0000 7fe529d9b640 1 -- 172.21.15.49:0/1934752134 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043268237 0 0) 0x7fe52c14ccc0 con 0x7fe52c148bc0 2024-03-14T21:14:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe523fff640 1 --2- 172.21.15.49:0/1934752134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c14bde0 0x7fe52c06ed60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe514009870 tx=0x7fe51402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b01262804f1363 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe529d9b640 1 -- 172.21.15.49:0/1934752134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51402f840 con 0x7fe52c14bde0 2024-03-14T21:14:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe529d9b640 1 -- 172.21.15.49:0/1934752134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51402f9a0 con 0x7fe52c14bde0 2024-03-14T21:14:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe529d9b640 1 -- 172.21.15.49:0/1934752134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51402fd30 con 0x7fe52c14bde0 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe530d63640 1 -- 172.21.15.49:0/1934752134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c14bde0 msgr2=0x7fe52c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe530d63640 1 --2- 172.21.15.49:0/1934752134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c14bde0 0x7fe52c06ed60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe514009870 tx=0x7fe51402ba20 comp rx=0 tx=0).stop 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe530d63640 1 -- 172.21.15.49:0/1934752134 shutdown_connections 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe530d63640 1 --2- 172.21.15.49:0/1934752134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c14bde0 0x7fe52c06ed60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe530d63640 1 -- 172.21.15.49:0/1934752134 >> 172.21.15.49:0/1934752134 conn(0x7fe52c06a310 msgr2=0x7fe52c071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe530d63640 1 -- 172.21.15.49:0/1934752134 shutdown_connections 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.465+0000 7fe530d63640 1 -- 172.21.15.49:0/1934752134 wait complete. 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.466+0000 7fe530d63640 1 Processor -- start 2024-03-14T21:14:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.466+0000 7fe530d63640 1 -- start start 2024-03-14T21:14:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe530d63640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe52c071b10 0x7fe52c147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe530d63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c148bc0 0x7fe52c0a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe530d63640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe52c14bde0 0x7fe52c141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe530d63640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c0a0e30 con 0x7fe52c148bc0 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe530d63640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c147640 con 0x7fe52c071b10 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe530d63640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c146880 con 0x7fe52c14bde0 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c148bc0 0x7fe52c0a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c148bc0 0x7fe52c0a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43962/0 (socket says 172.21.15.49:43962) 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 -- 172.21.15.49:0/621397688 learned_addr learned my addr 172.21.15.49:0/621397688 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe52ad9d640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe52c14bde0 0x7fe52c141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe52a59c640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe52c071b10 0x7fe52c147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 -- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe52c14bde0 msgr2=0x7fe52c141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe52c14bde0 0x7fe52c141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 -- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe52c071b10 msgr2=0x7fe52c147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe52c071b10 0x7fe52c147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.467+0000 7fe523fff640 1 -- 172.21.15.49:0/621397688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140094d0 con 0x7fe52c148bc0 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe52a59c640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe52c071b10 0x7fe52c147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe52ad9d640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe52c14bde0 0x7fe52c141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe523fff640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c148bc0 0x7fe52c0a16e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe514009870 tx=0x7fe514004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe522ffd640 1 -- 172.21.15.49:0/621397688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514037d10 con 0x7fe52c148bc0 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe522ffd640 1 -- 172.21.15.49:0/621397688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe514037e70 con 0x7fe52c148bc0 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c146b60 con 0x7fe52c148bc0 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe522ffd640 1 -- 172.21.15.49:0/621397688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514041620 con 0x7fe52c148bc0 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.468+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c147120 con 0x7fe52c148bc0 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.469+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52c154100 con 0x7fe52c148bc0 2024-03-14T21:14:14.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.469+0000 7fe522ffd640 1 -- 172.21.15.49:0/621397688 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe514041780 con 0x7fe52c148bc0 2024-03-14T21:14:14.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.470+0000 7fe522ffd640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe50c06a7a0 0x7fe50c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.470+0000 7fe522ffd640 1 -- 172.21.15.49:0/621397688 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe51403c070 con 0x7fe52c148bc0 2024-03-14T21:14:14.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.470+0000 7fe52a59c640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe50c06a7a0 0x7fe50c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.470+0000 7fe52a59c640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe50c06a7a0 0x7fe50c06cc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe51c003eb0 tx=0x7fe51c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.474+0000 7fe522ffd640 1 -- 172.21.15.49:0/621397688 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe52c154100 con 0x7fe52c148bc0 2024-03-14T21:14:14.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.656+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe52c1469c0 con 0x7fe52c148bc0 2024-03-14T21:14:14.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.656+0000 7fe522ffd640 1 -- 172.21.15.49:0/621397688 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe52c1469c0 con 0x7fe52c148bc0 2024-03-14T21:14:14.639 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-14T21:14:14.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe50c06a7a0 msgr2=0x7fe50c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe50c06a7a0 0x7fe50c06cc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe51c003eb0 tx=0x7fe51c009040 comp rx=0 tx=0).stop 2024-03-14T21:14:14.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c148bc0 msgr2=0x7fe52c0a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c148bc0 0x7fe52c0a16e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe514009870 tx=0x7fe514004730 comp rx=0 tx=0).stop 2024-03-14T21:14:14.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 shutdown_connections 2024-03-14T21:14:14.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe50c06a7a0 0x7fe50c06cc60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe52c14bde0 0x7fe52c141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe52c148bc0 0x7fe52c0a16e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 --2- 172.21.15.49:0/621397688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe52c071b10 0x7fe52c147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 >> 172.21.15.49:0/621397688 conn(0x7fe52c06a310 msgr2=0x7fe52c06eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:14.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 shutdown_connections 2024-03-14T21:14:14.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.658+0000 7fe530d63640 1 -- 172.21.15.49:0/621397688 wait complete. 2024-03-14T21:14:14.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T21:14:14.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.775+0000 7f7076d3e640 1 Processor -- start 2024-03-14T21:14:14.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.775+0000 7f7076d3e640 1 -- start start 2024-03-14T21:14:14.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f7076d3e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f707014ae10 0x7f707014b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f7076d3e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707006aeb0 con 0x7f70700a5710 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f7076d3e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70700bf980 con 0x7f70700bae90 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f7076d3e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70700bfe10 con 0x7f707014ae10 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f706ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f707014ae10 0x7f707014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f70752db640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f70700a5710 0x7f7070155640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47006/0 (socket says 172.21.15.49:47006) 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f70752db640 1 -- 172.21.15.49:0/3252402478 learned_addr learned my addr 172.21.15.49:0/3252402478 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f706ffff640 1 -- 172.21.15.49:0/3252402478 >> v1:172.21.15.97:6789/0 conn(0x7f70700bae90 legacy=0x7f70700bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f706f7fe640 1 -- 172.21.15.49:0/3252402478 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432930657 0 0) 0x7f707006aeb0 con 0x7f70700a5710 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f706ffff640 1 -- 172.21.15.49:0/3252402478 >> v1:172.21.15.49:6789/0 conn(0x7f70700a5710 legacy=0x7f7070155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.776+0000 7f706ffff640 1 -- 172.21.15.49:0/3252402478 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700bc210 con 0x7f707014ae10 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.777+0000 7f706ffff640 1 --2- 172.21.15.49:0/3252402478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014b230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70640097a0 tx=0x7f706402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbf0ca1e8b40f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.777+0000 7f706f7fe640 1 -- 172.21.15.49:0/3252402478 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064031830 con 0x7f707014ae10 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.777+0000 7f706f7fe640 1 -- 172.21.15.49:0/3252402478 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7064031990 con 0x7f707014ae10 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.777+0000 7f706f7fe640 1 -- 172.21.15.49:0/3252402478 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064031d20 con 0x7f707014ae10 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.778+0000 7f7076d3e640 1 -- 172.21.15.49:0/3252402478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 msgr2=0x7f707014b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.778+0000 7f7076d3e640 1 --2- 172.21.15.49:0/3252402478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014b230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70640097a0 tx=0x7f706402b8d0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.778+0000 7f7076d3e640 1 -- 172.21.15.49:0/3252402478 shutdown_connections 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.778+0000 7f7076d3e640 1 --2- 172.21.15.49:0/3252402478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014b230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.778+0000 7f7076d3e640 1 -- 172.21.15.49:0/3252402478 >> 172.21.15.49:0/3252402478 conn(0x7f70700c02a0 msgr2=0x7f70700c06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.778+0000 7f7076d3e640 1 -- 172.21.15.49:0/3252402478 shutdown_connections 2024-03-14T21:14:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.778+0000 7f7076d3e640 1 -- 172.21.15.49:0/3252402478 wait complete. 2024-03-14T21:14:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.779+0000 7f7076d3e640 1 Processor -- start 2024-03-14T21:14:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.779+0000 7f7076d3e640 1 -- start start 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70700a5710 0x7f70700b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70700bae90 0x7f707014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707014eb30 con 0x7f707014ae10 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707014e570 con 0x7f70700a5710 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707014e850 con 0x7f70700bae90 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43988/0 (socket says 172.21.15.49:43988) 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 -- 172.21.15.49:0/2116250483 learned_addr learned my addr 172.21.15.49:0/2116250483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f706ffff640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70700bae90 0x7f707014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7074ada640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70700a5710 0x7f70700b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 -- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70700bae90 msgr2=0x7f707014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70700bae90 0x7f707014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 -- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70700a5710 msgr2=0x7f70700b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70700a5710 0x7f70700b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 -- 172.21.15.49:0/2116250483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064009400 con 0x7f707014ae10 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f70752db640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014dcf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f706800c7e0 tx=0x7f706800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f706d7fa640 1 -- 172.21.15.49:0/2116250483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706800a640 con 0x7f707014ae10 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f706d7fa640 1 -- 172.21.15.49:0/2116250483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706800a7c0 con 0x7f707014ae10 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f706d7fa640 1 -- 172.21.15.49:0/2116250483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706800ab70 con 0x7f707014ae10 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707014d290 con 0x7f707014ae10 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.780+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7070149840 con 0x7f707014ae10 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.781+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707014d480 con 0x7f707014ae10 2024-03-14T21:14:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.782+0000 7f706d7fa640 1 -- 172.21.15.49:0/2116250483 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f706800acd0 con 0x7f707014ae10 2024-03-14T21:14:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.782+0000 7f706d7fa640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f705c06a7a0 0x7f705c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.782+0000 7f706d7fa640 1 -- 172.21.15.49:0/2116250483 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7068088390 con 0x7f707014ae10 2024-03-14T21:14:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.782+0000 7f7074ada640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f705c06a7a0 0x7f705c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.783+0000 7f7074ada640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f705c06a7a0 0x7f705c06cc60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7060000f60 tx=0x7f7060009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:14.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:14.787+0000 7f706d7fa640 1 -- 172.21.15.49:0/2116250483 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f70680537a0 con 0x7f707014ae10 2024-03-14T21:14:14.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.002+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7070066bb0 con 0x7f707014ae10 2024-03-14T21:14:14.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.002+0000 7f706d7fa640 1 -- 172.21.15.49:0/2116250483 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3969 (secure 0 0 0) 0x7f7068018090 con 0x7f707014ae10 2024-03-14T21:14:14.986 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:14.986 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2343109927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2343109927},{"type":"v1","addr":"172.21.15.97:6835","nonce":2343109927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1491968682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1491968682},{"type":"v1","addr":"172.21.15.49:6833","nonce":1491968682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T21:14:03.012761+0000","modified":"2024-03-14T21:14:05.655989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T21:14:14.986 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f705c06a7a0 msgr2=0x7f705c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f705c06a7a0 0x7f705c06cc60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7060000f60 tx=0x7f7060009040 comp rx=0 tx=0).stop 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 msgr2=0x7f707014dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014dcf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f706800c7e0 tx=0x7f706800cca0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 shutdown_connections 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f705c06a7a0 0x7f705c06cc60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f707014ae10 0x7f707014dcf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70700bae90 0x7f707014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 --2- 172.21.15.49:0/2116250483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70700a5710 0x7f70700b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 >> 172.21.15.49:0/2116250483 conn(0x7f70700c02a0 msgr2=0x7f70700bb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 shutdown_connections 2024-03-14T21:14:14.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.004+0000 7f7076d3e640 1 -- 172.21.15.49:0/2116250483 wait complete. 2024-03-14T21:14:14.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T21:14:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.121+0000 7f0696099640 1 Processor -- start 2024-03-14T21:14:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.121+0000 7f0696099640 1 -- start start 2024-03-14T21:14:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f0696099640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f06900baef0 0x7f06901480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f0696099640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069006a6e0 con 0x7f069006b620 2024-03-14T21:14:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f0696099640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06900bc2b0 con 0x7f06900bc940 2024-03-14T21:14:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f0696099640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069006ad10 con 0x7f06900baef0 2024-03-14T21:14:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f06900baef0 0x7f06901480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068effd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f069006b620 0x7f06900bf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47018/0 (socket says 172.21.15.49:47018) 2024-03-14T21:14:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f06900baef0 0x7f06901480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44002/0 (socket says 172.21.15.49:44002) 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068effd640 1 -- 172.21.15.49:0/1413323022 learned_addr learned my addr 172.21.15.49:0/1413323022 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068f7fe640 1 -- 172.21.15.49:0/1413323022 >> v1:172.21.15.97:6789/0 conn(0x7f06900bc940 legacy=0x7f0690155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068f7fe640 1 -- 172.21.15.49:0/1413323022 >> v1:172.21.15.49:6789/0 conn(0x7f069006b620 legacy=0x7f06900bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068f7fe640 1 -- 172.21.15.49:0/1413323022 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069006bab0 con 0x7f06900baef0 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.122+0000 7f068e7fc640 1 -- 172.21.15.49:0/1413323022 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083525773 0 0) 0x7f069006a6e0 con 0x7f069006b620 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.123+0000 7f068f7fe640 1 --2- 172.21.15.49:0/1413323022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900baef0 0x7f06901480b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06800097a0 tx=0x7f068002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b321d73f18f03a25 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.123+0000 7f068e7fc640 1 -- 172.21.15.49:0/1413323022 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068002e070 con 0x7f06900baef0 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.123+0000 7f068e7fc640 1 -- 172.21.15.49:0/1413323022 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06800315a0 con 0x7f06900baef0 2024-03-14T21:14:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.123+0000 7f068e7fc640 1 -- 172.21.15.49:0/1413323022 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680031930 con 0x7f06900baef0 2024-03-14T21:14:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.124+0000 7f0696099640 1 -- 172.21.15.49:0/1413323022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900baef0 msgr2=0x7f06901480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.124+0000 7f0696099640 1 --2- 172.21.15.49:0/1413323022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900baef0 0x7f06901480b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06800097a0 tx=0x7f068002b840 comp rx=0 tx=0).stop 2024-03-14T21:14:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.124+0000 7f0696099640 1 -- 172.21.15.49:0/1413323022 shutdown_connections 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.124+0000 7f0696099640 1 --2- 172.21.15.49:0/1413323022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900baef0 0x7f06901480b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.124+0000 7f0696099640 1 -- 172.21.15.49:0/1413323022 >> 172.21.15.49:0/1413323022 conn(0x7f06900bcd80 msgr2=0x7f06900bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.124+0000 7f0696099640 1 -- 172.21.15.49:0/1413323022 shutdown_connections 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.124+0000 7f0696099640 1 -- 172.21.15.49:0/1413323022 wait complete. 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 Processor -- start 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 -- start start 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06900baef0 0x7f0690146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900bc940 0x7f06901e3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06900b6a40 0x7f06900b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690146a70 con 0x7f06900bc940 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069014ed90 con 0x7f06900b6a40 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.125+0000 7f0696099640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06901e3210 con 0x7f06900baef0 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900bc940 0x7f06901e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900bc940 0x7f06901e3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44018/0 (socket says 172.21.15.49:44018) 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 -- 172.21.15.49:0/2451393075 learned_addr learned my addr 172.21.15.49:0/2451393075 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068f7fe640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06900baef0 0x7f0690146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068ffff640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06900b6a40 0x7f06900b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 -- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06900baef0 msgr2=0x7f0690146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06900baef0 0x7f0690146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 -- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06900b6a40 msgr2=0x7f06900b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06900b6a40 0x7f06900b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 -- 172.21.15.49:0/2451393075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0680009400 con 0x7f06900bc940 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068f7fe640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06900baef0 0x7f0690146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068ffff640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06900b6a40 0x7f06900b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f068effd640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900bc940 0x7f06901e3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f068400c4d0 tx=0x7f068400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f0677fff640 1 -- 172.21.15.49:0/2451393075 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06840074e0 con 0x7f06900bc940 2024-03-14T21:14:15.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f0677fff640 1 -- 172.21.15.49:0/2451393075 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0684007660 con 0x7f06900bc940 2024-03-14T21:14:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069014bfd0 con 0x7f06900bc940 2024-03-14T21:14:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f0677fff640 1 -- 172.21.15.49:0/2451393075 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684004a20 con 0x7f06900bc940 2024-03-14T21:14:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.126+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069014eab0 con 0x7f06900bc940 2024-03-14T21:14:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.127+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0690066bb0 con 0x7f06900bc940 2024-03-14T21:14:15.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.128+0000 7f0677fff640 1 -- 172.21.15.49:0/2451393075 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0684004b80 con 0x7f06900bc940 2024-03-14T21:14:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.128+0000 7f0677fff640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f067c06a7a0 0x7f067c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.128+0000 7f0677fff640 1 -- 172.21.15.49:0/2451393075 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06840882a0 con 0x7f06900bc940 2024-03-14T21:14:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.128+0000 7f068f7fe640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f067c06a7a0 0x7f067c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.129+0000 7f068f7fe640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f067c06a7a0 0x7f067c06cc60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0680036000 tx=0x7f068003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:15.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.132+0000 7f0677fff640 1 -- 172.21.15.49:0/2451393075 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f06840536c0 con 0x7f06900bc940 2024-03-14T21:14:15.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.312+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06901478b0 con 0x7f06900bc940 2024-03-14T21:14:15.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.314+0000 7f0677fff640 1 -- 172.21.15.49:0/2451393075 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f0684056ce0 con 0x7f06900bc940 2024-03-14T21:14:15.297 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:15.297 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":21,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:04.014218+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f067c06a7a0 msgr2=0x7f067c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f067c06a7a0 0x7f067c06cc60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0680036000 tx=0x7f068003d040 comp rx=0 tx=0).stop 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900bc940 msgr2=0x7f06901e3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900bc940 0x7f06901e3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f068400c4d0 tx=0x7f068400c990 comp rx=0 tx=0).stop 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 shutdown_connections 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f067c06a7a0 0x7f067c06cc60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06900b6a40 0x7f06900b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06900bc940 0x7f06901e3940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 --2- 172.21.15.49:0/2451393075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06900baef0 0x7f0690146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.315+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 >> 172.21.15.49:0/2451393075 conn(0x7f06900bcd80 msgr2=0x7f06900b9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:15.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.316+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 shutdown_connections 2024-03-14T21:14:15.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.316+0000 7f0696099640 1 -- 172.21.15.49:0/2451393075 wait complete. 2024-03-14T21:14:15.310 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T21:14:15.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc03a548640 1 Processor -- start 2024-03-14T21:14:15.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc03a548640 1 -- start start 2024-03-14T21:14:15.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc03a548640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc034141ae0 0x7fc03406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc03a548640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc034148360 0x7fc03406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc03a548640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc034147ed0 con 0x7fc03406d170 2024-03-14T21:14:15.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc03a548640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03406d9c0 con 0x7fc034148360 2024-03-14T21:14:15.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc03a548640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc034147690 con 0x7fc034141ae0 2024-03-14T21:14:15.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc0337fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc034148360 0x7fc03406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc038ae5640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc03406d170 0x7fc03414abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47034/0 (socket says 172.21.15.49:47034) 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc0337fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc034148360 0x7fc03406cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44022/0 (socket says 172.21.15.49:44022) 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.433+0000 7fc038ae5640 1 -- 172.21.15.49:0/2444365918 learned_addr learned my addr 172.21.15.49:0/2444365918 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.434+0000 7fc033fff640 1 --2- 172.21.15.49:0/2444365918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc034141ae0 0x7fc03406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.434+0000 7fc0337fe640 1 -- 172.21.15.49:0/2444365918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc034141ae0 msgr2=0x7fc03406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.434+0000 7fc0337fe640 1 --2- 172.21.15.49:0/2444365918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc034141ae0 0x7fc03406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.434+0000 7fc0337fe640 1 -- 172.21.15.49:0/2444365918 >> v1:172.21.15.49:6789/0 conn(0x7fc03406d170 legacy=0x7fc03414abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.434+0000 7fc0337fe640 1 -- 172.21.15.49:0/2444365918 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034147880 con 0x7fc034148360 2024-03-14T21:14:15.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.434+0000 7fc032ffd640 1 -- 172.21.15.49:0/2444365918 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675511579 0 0) 0x7fc034147ed0 con 0x7fc03406d170 2024-03-14T21:14:15.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc0337fe640 1 --2- 172.21.15.49:0/2444365918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc03406cc30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc02400ba40 tx=0x7fc02402e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cfb097c16892fa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc032ffd640 1 -- 172.21.15.49:0/2444365918 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02402ee30 con 0x7fc034148360 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc032ffd640 1 -- 172.21.15.49:0/2444365918 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024036440 con 0x7fc034148360 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc032ffd640 1 -- 172.21.15.49:0/2444365918 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240367d0 con 0x7fc034148360 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 -- 172.21.15.49:0/2444365918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 msgr2=0x7fc03406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 --2- 172.21.15.49:0/2444365918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc03406cc30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc02400ba40 tx=0x7fc02402e080 comp rx=0 tx=0).stop 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 -- 172.21.15.49:0/2444365918 shutdown_connections 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 --2- 172.21.15.49:0/2444365918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc03406cc30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 --2- 172.21.15.49:0/2444365918 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc034141ae0 0x7fc03406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 -- 172.21.15.49:0/2444365918 >> 172.21.15.49:0/2444365918 conn(0x7fc0340b9d30 msgr2=0x7fc0340ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 -- 172.21.15.49:0/2444365918 shutdown_connections 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.435+0000 7fc03a548640 1 -- 172.21.15.49:0/2444365918 wait complete. 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.436+0000 7fc03a548640 1 Processor -- start 2024-03-14T21:14:15.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.436+0000 7fc03a548640 1 -- start start 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc03a548640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034141ae0 0x7fc03413ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc03a548640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc0340bc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc03a548640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0340b2360 0x7fc03413f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc03a548640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03413b090 con 0x7fc034148360 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc03a548640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0341e3d50 con 0x7fc034141ae0 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc03a548640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0341e41e0 con 0x7fc0340b2360 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc0340bc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc0340bc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44030/0 (socket says 172.21.15.49:44030) 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 -- 172.21.15.49:0/3171607752 learned_addr learned my addr 172.21.15.49:0/3171607752 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc038ae5640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0340b2360 0x7fc03413f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc033fff640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034141ae0 0x7fc03413ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 -- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0340b2360 msgr2=0x7fc03413f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0340b2360 0x7fc03413f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 -- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034141ae0 msgr2=0x7fc03413ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034141ae0 0x7fc03413ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc0337fe640 1 -- 172.21.15.49:0/3171607752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02400b6a0 con 0x7fc034148360 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.437+0000 7fc033fff640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034141ae0 0x7fc03413ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.438+0000 7fc0337fe640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc0340bc790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc024039040 tx=0x7fc0240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.438+0000 7fc030ff9640 1 -- 172.21.15.49:0/3171607752 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240348d0 con 0x7fc034148360 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.438+0000 7fc030ff9640 1 -- 172.21.15.49:0/3171607752 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024034a50 con 0x7fc034148360 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.438+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0341e3420 con 0x7fc034148360 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.438+0000 7fc030ff9640 1 -- 172.21.15.49:0/3171607752 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024034e00 con 0x7fc034148360 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.438+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0341e39e0 con 0x7fc034148360 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.439+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0341e3b70 con 0x7fc034148360 2024-03-14T21:14:15.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.439+0000 7fc030ff9640 1 -- 172.21.15.49:0/3171607752 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc02403d070 con 0x7fc034148360 2024-03-14T21:14:15.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.440+0000 7fc030ff9640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc02006a7d0 0x7fc02006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:15.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.440+0000 7fc030ff9640 1 -- 172.21.15.49:0/3171607752 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0240ab170 con 0x7fc034148360 2024-03-14T21:14:15.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.440+0000 7fc033fff640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc02006a7d0 0x7fc02006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:15.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.440+0000 7fc033fff640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc02006a7d0 0x7fc02006cc90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc028006fd0 tx=0x7fc028008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:15.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.444+0000 7fc030ff9640 1 -- 172.21.15.49:0/3171607752 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc024076620 con 0x7fc034148360 2024-03-14T21:14:15.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.664+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0341e3ec0 con 0x7fc034148360 2024-03-14T21:14:15.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.665+0000 7fc030ff9640 1 -- 172.21.15.49:0/3171607752 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3969 (secure 0 0 0) 0x7fc024079c40 con 0x7fc034148360 2024-03-14T21:14:15.649 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T21:14:15.649 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2343109927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2343109927},{"type":"v1","addr":"172.21.15.97:6835","nonce":2343109927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1491968682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1491968682},{"type":"v1","addr":"172.21.15.49:6833","nonce":1491968682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T21:14:03.012761+0000","modified":"2024-03-14T21:14:05.655989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T21:14:15.649 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-14T21:14:15.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc02006a7d0 msgr2=0x7fc02006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc02006a7d0 0x7fc02006cc90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc028006fd0 tx=0x7fc028008040 comp rx=0 tx=0).stop 2024-03-14T21:14:15.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 msgr2=0x7fc0340bc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:15.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc0340bc790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc024039040 tx=0x7fc0240046c0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 shutdown_connections 2024-03-14T21:14:15.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc02006a7d0 0x7fc02006cc90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0340b2360 0x7fc03413f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc034148360 0x7fc0340bc790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 --2- 172.21.15.49:0/3171607752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc034141ae0 0x7fc03413ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:15.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 >> 172.21.15.49:0/3171607752 conn(0x7fc0340b9d30 msgr2=0x7fc0340b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:15.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 shutdown_connections 2024-03-14T21:14:15.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:15.667+0000 7fc03a548640 1 -- 172.21.15.49:0/3171607752 wait complete. 2024-03-14T21:14:15.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T21:14:03.012761+0000', 'modified': '2024-03-14T21:14:05.655989+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6835/403429269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 403429269}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 403429269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T21:14:15.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T21:14:15.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-14T21:14:16.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.089+0000 7fa49f27a640 1 Processor -- start 2024-03-14T21:14:16.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49f27a640 1 -- start start 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49f27a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa49814b660 0x7fa49806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49f27a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b90b0 0x7fa498155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49f27a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49806e540 con 0x7fa4980b8c40 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49f27a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49814e090 con 0x7fa49814b660 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49f27a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980b9970 con 0x7fa4980b90b0 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49d016640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa49814b660 0x7fa49806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49c815640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa4980b8c40 0x7fa49814e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47042/0 (socket says 172.21.15.49:47042) 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.090+0000 7fa49c815640 1 -- 172.21.15.49:0/3570103125 learned_addr learned my addr 172.21.15.49:0/3570103125 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.091+0000 7fa49d817640 1 --2- 172.21.15.49:0/3570103125 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b90b0 0x7fa498155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.091+0000 7fa49d016640 1 -- 172.21.15.49:0/3570103125 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b90b0 msgr2=0x7fa498155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.091+0000 7fa49d016640 1 --2- 172.21.15.49:0/3570103125 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b90b0 0x7fa498155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.091+0000 7fa49d016640 1 -- 172.21.15.49:0/3570103125 >> v1:172.21.15.49:6789/0 conn(0x7fa4980b8c40 legacy=0x7fa49814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.091+0000 7fa48ffff640 1 -- 172.21.15.49:0/3570103125 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755314825 0 0) 0x7fa49806e540 con 0x7fa4980b8c40 2024-03-14T21:14:16.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.091+0000 7fa49d016640 1 -- 172.21.15.49:0/3570103125 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980703c0 con 0x7fa49814b660 2024-03-14T21:14:16.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49d016640 1 --2- 172.21.15.49:0/3570103125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49806e000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097a0 tx=0x7fa49402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f866d9cc6fb097 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa48ffff640 1 -- 172.21.15.49:0/3570103125 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494030610 con 0x7fa49814b660 2024-03-14T21:14:16.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa48ffff640 1 -- 172.21.15.49:0/3570103125 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494030770 con 0x7fa49814b660 2024-03-14T21:14:16.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa48ffff640 1 -- 172.21.15.49:0/3570103125 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494030b00 con 0x7fa49814b660 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 -- 172.21.15.49:0/3570103125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 msgr2=0x7fa49806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3570103125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49806e000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097a0 tx=0x7fa49402b960 comp rx=0 tx=0).stop 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 -- 172.21.15.49:0/3570103125 shutdown_connections 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3570103125 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b90b0 0x7fa498155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3570103125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49806e000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 -- 172.21.15.49:0/3570103125 >> 172.21.15.49:0/3570103125 conn(0x7fa4980b9d30 msgr2=0x7fa4980ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 -- 172.21.15.49:0/3570103125 shutdown_connections 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.092+0000 7fa49f27a640 1 -- 172.21.15.49:0/3570103125 wait complete. 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.093+0000 7fa49f27a640 1 Processor -- start 2024-03-14T21:14:16.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.093+0000 7fa49f27a640 1 -- start start 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49f27a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b8c40 0x7fa4980bc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49f27a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4980b90b0 0x7fa4981dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49f27a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49809f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49f27a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4981e0da0 con 0x7fa49814b660 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49f27a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980bcd40 con 0x7fa4980b90b0 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49f27a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4981e34b0 con 0x7fa4980b8c40 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49809f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49809f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44056/0 (socket says 172.21.15.49:44056) 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 -- 172.21.15.49:0/3272570396 learned_addr learned my addr 172.21.15.49:0/3272570396 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d016640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b8c40 0x7fa4980bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49c815640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4980b90b0 0x7fa4981dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 -- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b8c40 msgr2=0x7fa4980bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b8c40 0x7fa4980bc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 -- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4980b90b0 msgr2=0x7fa4981dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4980b90b0 0x7fa4981dfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d817640 1 -- 172.21.15.49:0/3272570396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494009400 con 0x7fa49814b660 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49c815640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4980b90b0 0x7fa4981dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.094+0000 7fa49d016640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b8c40 0x7fa4980bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.095+0000 7fa49d817640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49809f390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa490007e40 tx=0x7fa49000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.095+0000 7fa48dffb640 1 -- 172.21.15.49:0/3272570396 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490014620 con 0x7fa49814b660 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.095+0000 7fa48dffb640 1 -- 172.21.15.49:0/3272570396 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49000ee90 con 0x7fa49814b660 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.095+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4981e4230 con 0x7fa49814b660 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.095+0000 7fa48dffb640 1 -- 172.21.15.49:0/3272570396 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4900149e0 con 0x7fa49814b660 2024-03-14T21:14:16.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.095+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4981e3750 con 0x7fa49814b660 2024-03-14T21:14:16.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.096+0000 7fa48dffb640 1 -- 172.21.15.49:0/3272570396 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa490014b40 con 0x7fa49814b660 2024-03-14T21:14:16.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.097+0000 7fa48dffb640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa47c06a7a0 0x7fa47c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.097+0000 7fa48dffb640 1 -- 172.21.15.49:0/3272570396 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4900896e0 con 0x7fa49814b660 2024-03-14T21:14:16.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.097+0000 7fa49d016640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa47c06a7a0 0x7fa47c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.098+0000 7fa49d016640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa47c06a7a0 0x7fa47c06cc60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa494036000 tx=0x7fa494008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.098+0000 7fa4a1dea740 1 Processor -- start 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.098+0000 7fa4a1dea740 1 -- start start 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651b58f0 0x562f651e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f651e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f65115e20 con 0x562f651b58f0 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f651251d0 con 0x562f651f11e0 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f65122960 con 0x562f651e6ac0 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651b58f0 0x562f651e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651b58f0 0x562f651e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44064/0 (socket says 172.21.15.49:44064) 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 -- 172.21.15.49:0/678985748 learned_addr learned my addr 172.21.15.49:0/678985748 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49d016640 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49d817640 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f651e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 -- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 msgr2=0x562f651e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f651e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 -- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 msgr2=0x562f651b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 -- 172.21.15.49:0/678985748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f650aab10 con 0x562f651b58f0 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa49c815640 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651b58f0 0x562f651e6580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa49806e6d0 tx=0x7fa488001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52ebf5c2b7321d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa482ffd640 1 -- 172.21.15.49:0/678985748 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488005ac0 con 0x562f651b58f0 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa482ffd640 1 -- 172.21.15.49:0/678985748 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa488005c20 con 0x562f651b58f0 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.099+0000 7fa482ffd640 1 -- 172.21.15.49:0/678985748 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4880070d0 con 0x562f651b58f0 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 -- 172.21.15.49:0/678985748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651b58f0 msgr2=0x562f651e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651b58f0 0x562f651e6580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa49806e6d0 tx=0x7fa488001b50 comp rx=0 tx=0).stop 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 -- 172.21.15.49:0/678985748 shutdown_connections 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f651e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651b58f0 0x562f651e6580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/678985748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 -- 172.21.15.49:0/678985748 >> 172.21.15.49:0/678985748 conn(0x562f651b6d40 msgr2=0x562f651f08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 -- 172.21.15.49:0/678985748 shutdown_connections 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 -- 172.21.15.49:0/678985748 wait complete. 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.100+0000 7fa4a1dea740 1 -- start start 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 1 Processor -- start 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6511e250 con 0x562f651e6ac0 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f650dc8b0 con 0x562f651f11e0 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f650dc550 con 0x562f651e5cd0 2024-03-14T21:14:16.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa4a1dea740 10 client.0 _ll_register_callbacks cb 0x562f6510eda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa49d016640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.101+0000 7fa49d016640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44080/0 (socket says 172.21.15.49:44080) 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d016640 1 -- 172.21.15.49:0/1179603763 learned_addr learned my addr 172.21.15.49:0/1179603763 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49c815640 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d817640 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d016640 1 -- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 msgr2=0x562f65200260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d016640 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d016640 1 -- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 msgr2=0x562f651e12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d016640 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651e12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d016640 1 -- 172.21.15.49:0/1179603763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48800f040 con 0x562f651e6ac0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa49d016640 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e0db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa494009890 tx=0x7fa49402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68036bdd5130f7d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa477fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa477fff640 1 -- 172.21.15.49:0/1179603763 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49404d050 con 0x562f651e6ac0 2024-03-14T21:14:16.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa477fff640 1 -- 172.21.15.49:0/1179603763 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa49403a460 con 0x562f651e6ac0 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f64e123c0 con 0x562f651e6ac0 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa477fff640 1 -- 172.21.15.49:0/1179603763 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa494030e50 con 0x562f651e6ac0 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa4a1dea740 10 client.4650 fetch_fsmap learned FSMap version 7 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x562f6511c8d0 con 0x562f651e6ac0 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa477fff640 1 -- 172.21.15.49:0/1179603763 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f64e123c0 con 0x562f651e6ac0 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa477fff640 1 -- 172.21.15.49:0/1179603763 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 7) v1 ==== 2569+0+0 (secure 0 0 0) 0x7fa494043070 con 0x562f651e6ac0 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.102+0000 7fa4a1dea740 10 client.4650 fetch_fsmap finished waiting for FSMap version 7 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa4a1dea740 10 client.4650 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa4a1dea740 20 client.4650 populate_metadata read hostname 'smithi049' 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f65200870 0x562f65200c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa4a1dea740 4 client.4650 mds_command: new command op to 4272 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 --> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f64e236c0 con 0x562f65200870 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa49c815640 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f65200870 0x562f65200c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa477fff640 10 client.4650 ms_handle_connect on v2:172.21.15.49:6834/403429269 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.103+0000 7fa49c815640 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f65200870 0x562f65200c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f60af0ecba3bdb server_cookie=bd4e1bfb30e89892 in_seq=0 out_seq=0 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa477fff640 1 -- 172.21.15.49:0/1179603763 <== mds.0 v2:172.21.15.49:6834/403429269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562f64e236c0 con 0x562f65200870 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa477fff640 10 client.4650 handle_command_reply: tid=0 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 client.4650 shutdown 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 msgr2=0x562f651e0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e0db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa494009890 tx=0x7fa49402e000 comp rx=0 tx=0).stop 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 shutdown_connections 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f65200870 0x562f65200c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa477fff640 0 client.4650 ms_handle_reset on v2:172.21.15.49:6834/403429269 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1179603763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e0db0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 >> 172.21.15.49:0/1179603763 conn(0x562f651b58f0 msgr2=0x562f651e23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 shutdown_connections 2024-03-14T21:14:16.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1179603763 wait complete. 2024-03-14T21:14:16.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.104+0000 7fa4a1dea740 20 client.4650 trim_cache size 0 max 16384 2024-03-14T21:14:16.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.119+0000 7fa4a1dea740 1 Processor -- start 2024-03-14T21:14:16.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.119+0000 7fa4a1dea740 1 -- start start 2024-03-14T21:14:16.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f651b7420 con 0x562f651e6ac0 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f650a22a0 con 0x562f651f11e0 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f6520fae0 con 0x562f651e5cd0 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44096/0 (socket says 172.21.15.49:44096) 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 -- 172.21.15.49:0/1344457925 learned_addr learned my addr 172.21.15.49:0/1344457925 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49d817640 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49d016640 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 -- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 msgr2=0x562f65200020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 -- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 msgr2=0x562f651ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651ec360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 -- 172.21.15.49:0/1344457925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f650bec50 con 0x562f651e6ac0 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49d016640 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49d817640 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa49c815640 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e9ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa488007fd0 tx=0x7fa48800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa4f15d1b30fa83 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4837fe640 1 -- 172.21.15.49:0/1344457925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488024070 con 0x562f651e6ac0 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4837fe640 1 -- 172.21.15.49:0/1344457925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa488001330 con 0x562f651e6ac0 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.120+0000 7fa4837fe640 1 -- 172.21.15.49:0/1344457925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488007250 con 0x562f651e6ac0 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1344457925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 msgr2=0x562f651e9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e9ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa488007fd0 tx=0x7fa48800a000 comp rx=0 tx=0).stop 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1344457925 shutdown_connections 2024-03-14T21:14:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651f11e0 0x562f651ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651e6ac0 0x562f651e9ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/1344457925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e5cd0 0x562f65200020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1344457925 >> 172.21.15.49:0/1344457925 conn(0x562f651b58f0 msgr2=0x562f651b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1344457925 shutdown_connections 2024-03-14T21:14:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 -- 172.21.15.49:0/1344457925 wait complete. 2024-03-14T21:14:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.121+0000 7fa4a1dea740 1 -- start start 2024-03-14T21:14:16.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 1 Processor -- start 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651e5cd0 0x562f651ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f652035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651f11e0 0x562f6524c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f65256470 con 0x562f651f11e0 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f651ec160 con 0x562f651e5cd0 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f651ec440 con 0x562f651e6ac0 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa4a1dea740 10 client.0 _ll_register_callbacks cb 0x7fa488007e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T21:14:16.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa49d817640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651f11e0 0x562f6524c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa49d817640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651f11e0 0x562f6524c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44108/0 (socket says 172.21.15.49:44108) 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa49d817640 1 -- 172.21.15.49:0/2236040716 learned_addr learned my addr 172.21.15.49:0/2236040716 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa49c815640 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f652035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.122+0000 7fa49d016640 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651e5cd0 0x562f651ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa49d817640 1 -- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 msgr2=0x562f652035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa49d817640 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f652035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa49d817640 1 -- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651e5cd0 msgr2=0x562f651ec8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa49d817640 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651e5cd0 0x562f651ec8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa49d817640 1 -- 172.21.15.49:0/2236040716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f650aab10 con 0x562f651f11e0 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa49d817640 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651f11e0 0x562f6524c740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c040 tx=0x7fa490058910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b0d1f0f48cb545 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa477fff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa477fff640 1 -- 172.21.15.49:0/2236040716 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490060c40 con 0x562f651f11e0 2024-03-14T21:14:16.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa477fff640 1 -- 172.21.15.49:0/2236040716 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa490056bf0 con 0x562f651f11e0 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f64e76d10 con 0x562f651f11e0 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa477fff640 1 -- 172.21.15.49:0/2236040716 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa490077170 con 0x562f651f11e0 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 10 client.4656 fetch_fsmap learned FSMap version 7 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x562f650e4360 con 0x562f651f11e0 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa477fff640 1 -- 172.21.15.49:0/2236040716 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f64e76d10 con 0x562f651f11e0 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa477fff640 1 -- 172.21.15.49:0/2236040716 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 7) v1 ==== 2569+0+0 (secure 0 0 0) 0x7fa490057d40 con 0x562f651f11e0 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 10 client.4656 fetch_fsmap finished waiting for FSMap version 7 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 10 client.4656 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 20 client.4656 populate_metadata read hostname 'smithi049' 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f651b8860 0x562f651b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 4 client.4656 mds_command: new command op to 4272 tid=0[{"prefix": "status"}] 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.123+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 --> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f651e22b0 con 0x562f651b8860 2024-03-14T21:14:16.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa49d016640 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f651b8860 0x562f651b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa477fff640 10 client.4656 ms_handle_connect on v2:172.21.15.49:6834/403429269 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa49d016640 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f651b8860 0x562f651b8c40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cb9cdf8021c22dd server_cookie=94b41fdda861f29a in_seq=0 out_seq=0 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa477fff640 1 -- 172.21.15.49:0/2236040716 <== mds.0 v2:172.21.15.49:6834/403429269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x562f651e22b0 con 0x562f651b8860 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa477fff640 10 client.4656 handle_command_reply: tid=0 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 client.4656 shutdown 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651f11e0 msgr2=0x562f6524c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651f11e0 0x562f6524c740 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c040 tx=0x7fa490058910 comp rx=0 tx=0).stop 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 shutdown_connections 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:6834/403429269,v1:172.21.15.49:6835/403429269] conn(0x562f651b8860 0x562f651b8c40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562f651f11e0 0x562f6524c740 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa477fff640 0 client.4656 ms_handle_reset on v2:172.21.15.49:6834/403429269 2024-03-14T21:14:16.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x562f651e6ac0 0x562f652035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 --2- 172.21.15.49:0/2236040716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f651e5cd0 0x562f651ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 >> 172.21.15.49:0/2236040716 conn(0x562f651b58f0 msgr2=0x562f651b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 shutdown_connections 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 1 -- 172.21.15.49:0/2236040716 wait complete. 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.124+0000 7fa4a1dea740 20 client.4656 trim_cache size 0 max 16384 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "25282add-1360-4a83-a2da-1a7e2b367f69", 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4272, 2024-03-14T21:14:16.115 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 13.110238106000001, 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 7, 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 21, 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 21, 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 19.639815452000001 2024-03-14T21:14:16.116 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa47c06a7a0 msgr2=0x7fa47c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa47c06a7a0 0x7fa47c06cc60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa494036000 tx=0x7fa494008030 comp rx=0 tx=0).stop 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 msgr2=0x7fa49809f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49809f390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa490007e40 tx=0x7fa49000e4b0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 shutdown_connections 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa47c06a7a0 0x7fa47c06cc60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa49814b660 0x7fa49809f390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4980b90b0 0x7fa4981dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 --2- 172.21.15.49:0/3272570396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4980b8c40 0x7fa4980bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 >> 172.21.15.49:0/3272570396 conn(0x7fa4980b9d30 msgr2=0x7fa49814e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 shutdown_connections 2024-03-14T21:14:16.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.125+0000 7fa49f27a640 1 -- 172.21.15.49:0/3272570396 wait complete. 2024-03-14T21:14:16.119 INFO:teuthology.run_tasks:Running task kclient... 2024-03-14T21:14:16.145 INFO:tasks.kclient:Mounting kernel clients... 2024-03-14T21:14:16.145 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-14T21:14:16.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:16.145 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-03-14T21:14:16.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:16.204 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-03-14T21:14:16.304 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-03-14T21:14:16.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T21:14:16.308 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-14T21:14:16.308 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-14T21:14:16.308 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:16.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T21:14:16.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.431+0000 7f8750b88640 1 Processor -- start 2024-03-14T21:14:16.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.431+0000 7f8750b88640 1 -- start start 2024-03-14T21:14:16.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8750b88640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 0x7f874c06a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8750b88640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f874c0bb1c0 0x7f874c14e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8750b88640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c0b3a20 con 0x7f874c0bad60 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8750b88640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c0ba550 con 0x7f874c0bb1c0 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8750b88640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0bfbe0 con 0x7f874c14b820 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8743fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f874c0bad60 0x7f874c14e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47056/0 (socket says 172.21.15.49:47056) 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8743fff640 1 -- 172.21.15.49:0/1174207869 learned_addr learned my addr 172.21.15.49:0/1174207869 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f874ad9d640 1 --2- 172.21.15.49:0/1174207869 >> v2:172.21.15.97:3300/0 conn(0x7f874c0bb1c0 0x7f874c14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472987064 0 0) 0x7f874c0b3a20 con 0x7f874c0bad60 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8738003520 con 0x7f874c0bad60 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.432+0000 7f874a59c640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 0x7f874c06a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1948370895 0 0) 0x7f8738003520 con 0x7f874c0bad60 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 msgr2=0x7f874c06a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 0x7f874c06a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 >> v2:172.21.15.97:3300/0 conn(0x7f874c0bb1c0 msgr2=0x7f874c14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 --2- 172.21.15.49:0/1174207869 >> v2:172.21.15.97:3300/0 conn(0x7f874c0bb1c0 0x7f874c14e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0bfec0 con 0x7f874c0bad60 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f8734001670 con 0x7f874c0bad60 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f874a59c640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 0x7f874c06a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 >> v1:172.21.15.49:6789/0 conn(0x7f874c0bad60 legacy=0x7f874c14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 0x7f8738003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8738003eb0 0x7f87380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8738004d00 0x7f8738005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0ba550 con 0x7f8738003eb0 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0bfbe0 con 0x7f8738004d00 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873800b9b0 con 0x7f874c14b820 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8743fff640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8738003eb0 0x7f87380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f874a59c640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 0x7f8738003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f874ad9d640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8738004d00 0x7f8738005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8743fff640 1 -- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 msgr2=0x7f8738003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8743fff640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f874c14b820 0x7f8738003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8743fff640 1 -- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8738004d00 msgr2=0x7f8738005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8743fff640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8738004d00 0x7f8738005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.433+0000 7f8743fff640 1 -- 172.21.15.49:0/1174207869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738003520 con 0x7f8738003eb0 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8743fff640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8738003eb0 0x7f87380047c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f873400c240 tx=0x7f87380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02925ca7a3e4dec server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873402cb00 con 0x7f8738003eb0 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873402cc60 con 0x7f8738003eb0 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8749d9b640 1 -- 172.21.15.49:0/1174207869 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734005af0 con 0x7f8738003eb0 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8750b88640 1 -- 172.21.15.49:0/1174207869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8738003eb0 msgr2=0x7f87380047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8750b88640 1 --2- 172.21.15.49:0/1174207869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8738003eb0 0x7f87380047c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f873400c240 tx=0x7f87380057f0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f874ad9d640 1 -- 172.21.15.49:0/1174207869 reap_dead start 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8750b88640 1 -- 172.21.15.49:0/1174207869 shutdown_connections 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8750b88640 1 -- 172.21.15.49:0/1174207869 >> 172.21.15.49:0/1174207869 conn(0x7f874c0bcd80 msgr2=0x7f874c0bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8750b88640 1 -- 172.21.15.49:0/1174207869 shutdown_connections 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.434+0000 7f8750b88640 1 -- 172.21.15.49:0/1174207869 wait complete. 2024-03-14T21:14:16.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 Processor -- start 2024-03-14T21:14:16.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 -- start start 2024-03-14T21:14:16.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8738003eb0 0x7f874c1e0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c0b3120 0x7f874c099fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f874c09b970 0x7f874c09aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c09bdf0 con 0x7f874c09b970 2024-03-14T21:14:16.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0b2eb0 con 0x7f874c0b3120 2024-03-14T21:14:16.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8750b88640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0b2bd0 con 0x7f8738003eb0 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f874ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f874c09b970 0x7f874c09aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f874ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f874c09b970 0x7f874c09aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44120/0 (socket says 172.21.15.49:44120) 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f874ad9d640 1 -- 172.21.15.49:0/4107862217 learned_addr learned my addr 172.21.15.49:0/4107862217 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f874a59c640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8738003eb0 0x7f874c1e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.436+0000 7f8743fff640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c0b3120 0x7f874c099fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f874ad9d640 1 -- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8738003eb0 msgr2=0x7f874c1e0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f874ad9d640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8738003eb0 0x7f874c1e0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f874ad9d640 1 -- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c0b3120 msgr2=0x7f874c099fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f874ad9d640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c0b3120 0x7f874c099fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f874ad9d640 1 -- 172.21.15.49:0/4107862217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c1dfc20 con 0x7f874c09b970 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f8743fff640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c0b3120 0x7f874c099fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f874ad9d640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f874c09b970 0x7f874c09aa80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f874c0b7240 tx=0x7f87440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f8742ffd640 1 -- 172.21.15.49:0/4107862217 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400e070 con 0x7f874c09b970 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f8742ffd640 1 -- 172.21.15.49:0/4107862217 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8744011e10 con 0x7f874c09b970 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f8742ffd640 1 -- 172.21.15.49:0/4107862217 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744003990 con 0x7f874c09b970 2024-03-14T21:14:16.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c1dff00 con 0x7f874c09b970 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.437+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c0b6d70 con 0x7f874c09b970 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.438+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c066bb0 con 0x7f874c09b970 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.438+0000 7f8742ffd640 1 -- 172.21.15.49:0/4107862217 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8744003af0 con 0x7f874c09b970 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.439+0000 7f8742ffd640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f872c06a7a0 0x7f872c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.439+0000 7f8742ffd640 1 -- 172.21.15.49:0/4107862217 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87440893d0 con 0x7f874c09b970 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.439+0000 7f874a59c640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f872c06a7a0 0x7f872c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.440+0000 7f874a59c640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f872c06a7a0 0x7f872c06cc60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f873c006fd0 tx=0x7f873c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T21:14:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.443+0000 7f8742ffd640 1 -- 172.21.15.49:0/4107862217 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8744054880 con 0x7f874c09b970 2024-03-14T21:14:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.660+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f874c06b470 con 0x7f874c09b970 2024-03-14T21:14:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.661+0000 7f8742ffd640 1 -- 172.21.15.49:0/4107862217 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8744057ea0 con 0x7f874c09b970 2024-03-14T21:14:16.644 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T21:14:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.662+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f872c06a7a0 msgr2=0x7f872c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.662+0000 7f8750b88640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f872c06a7a0 0x7f872c06cc60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f873c006fd0 tx=0x7f873c008040 comp rx=0 tx=0).stop 2024-03-14T21:14:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.662+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f874c09b970 msgr2=0x7f874c09aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T21:14:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.662+0000 7f8750b88640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f874c09b970 0x7f874c09aa80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f874c0b7240 tx=0x7f87440115a0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.662+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 shutdown_connections 2024-03-14T21:14:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.662+0000 7f8750b88640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f872c06a7a0 0x7f872c06cc60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.662+0000 7f8750b88640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f874c09b970 0x7f874c09aa80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.663+0000 7f8750b88640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874c0b3120 0x7f874c099fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.663+0000 7f8750b88640 1 --2- 172.21.15.49:0/4107862217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8738003eb0 0x7f874c1e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T21:14:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.663+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 >> 172.21.15.49:0/4107862217 conn(0x7f874c0bcd80 msgr2=0x7f874c09fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T21:14:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.663+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 shutdown_connections 2024-03-14T21:14:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T21:14:16.663+0000 7f8750b88640 1 -- 172.21.15.49:0/4107862217 wait complete. 2024-03-14T21:14:16.656 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-14T21:14:16.656 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T21:14:16.656 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-03-14T21:14:16.656 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T21:14:16.656 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:16.656 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-14T21:14:16.657 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-14T21:14:16.657 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T21:14:16.657 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T21:14:16.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:16.657 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-03-14T21:14:16.713 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T21:14:16.714 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42640sec preferred_lft 42640sec 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2024-03-14T21:14:16.715 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-03-14T21:14:16.716 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-14T21:14:16.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:16.716 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-14T21:14:16.716 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-14T21:14:16.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add name ceph-brx type bridge 2024-03-14T21:14:16.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr flush dev ceph-brx 2024-03-14T21:14:16.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set ceph-brx up 2024-03-14T21:14:16.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-14T21:14:16.717 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-14T21:14:17.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:17.162 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T21:14:17.235 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-03-14T21:14:17.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:17.242 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-03-14T21:14:17.298 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-03-14T21:14:17.298 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T21:14:17.298 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T21:14:17.298 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T21:14:17.299 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T21:14:17.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:17.299 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-14T21:14:17.299 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-14T21:14:17.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T21:14:17.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T21:14:17.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T21:14:17.299 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-14T21:14:17.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:17.946 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-03-14T21:14:18.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:18.000 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2024-03-14T21:14:18.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:18.047 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-14T21:14:18.047 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-14T21:14:18.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T21:14:18.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-14T21:14:18.047 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-14T21:14:18.169 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-14T21:14:18.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:18.169 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-14T21:14:18.169 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-14T21:14:18.169 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-14T21:14:18.169 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-14T21:14:18.169 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-14T21:14:18.169 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-14T21:14:18.170 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-14T21:14:18.170 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-14T21:14:18.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:18.432 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-14T21:14:18.432 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-14T21:14:18.432 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2024-03-14T21:14:18.432 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.0 master ceph-brx 2024-03-14T21:14:18.432 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-14T21:14:18.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T21:14:18.578 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:18.619 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T21:14:18.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T21:14:18.620 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:18.698 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-14T21:14:18.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:18.698 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-14T21:14:18.873 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-14T21:14:18.873 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-14T21:14:18.873 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-03-14T21:14:19.520 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-14T21:14:19.520 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-14T21:14:19.520 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-03-14T21:14:19.520 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.97:3300,172.21.15.97:3301" 2024-03-14T21:14:19.520 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T21:14:19.550 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-14T21:14:19.550 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-14T21:14:19.550 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2024-03-14T21:14:19.550 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.97:3300,172.21.15.97:3301" 2024-03-14T21:14:19.551 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T21:14:19.551 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.49:6789,172.21.15.97:6789,172.21.15.97:6790" 2024-03-14T21:14:19.551 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:6789,172.21.15.97:6789,172.21.15.97:6790:/ 2024-03-14T21:14:19.551 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=25282add-1360-4a83-a2da-1a7e2b367f69" will pass to kernel 2024-03-14T21:14:19.554 INFO:teuthology.orchestra.run.smithi163.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-14T21:14:19.554 INFO:teuthology.orchestra.run.smithi163.stdout: You just mounted a file system that supports labels which does not 2024-03-14T21:14:19.554 INFO:teuthology.orchestra.run.smithi163.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-14T21:14:19.555 INFO:teuthology.orchestra.run.smithi163.stdout: applications will generate AVC messages and not be allowed access to 2024-03-14T21:14:19.555 INFO:teuthology.orchestra.run.smithi163.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-14T21:14:19.559 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T21:14:19.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:19.560 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:19.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:19.655 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T21:14:19.726 INFO:teuthology.orchestra.run.smithi163.stdout:client4520 2024-03-14T21:14:19.726 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:14:19.727 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T21:14:19.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:19.780 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T21:14:19.852 INFO:teuthology.orchestra.run.smithi163.stdout:client4520 2024-03-14T21:14:19.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:19.852 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T21:14:19.922 INFO:teuthology.orchestra.run.smithi163.stdout:client4520 2024-03-14T21:14:19.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T21:14:19.922 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/25282add-1360-4a83-a2da-1a7e2b367f69.client4520/status') 2024-03-14T21:14:20.026 INFO:teuthology.orchestra.run.smithi163.stdout:instance: client.4520 (3)172.21.15.163:0/136364870 2024-03-14T21:14:20.026 INFO:teuthology.orchestra.run.smithi163.stdout:blocklisted: false 2024-03-14T21:14:20.026 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records in 2024-03-14T21:14:20.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records out 2024-03-14T21:14:20.027 INFO:teuthology.orchestra.run.smithi163.stderr:70 bytes copied, 0.000107416 s, 652 kB/s 2024-03-14T21:14:20.031 INFO:teuthology.run_tasks:Running task workunit... 2024-03-14T21:14:20.043 INFO:tasks.workunit:Pulling workunits from ref 234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:14:20.043 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-14T21:14:20.044 DEBUG:teuthology.orchestra.run.smithi163:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:20.099 INFO:teuthology.orchestra.run.smithi163.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-14 21:14:03.015285296 +0000 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-14 21:14:19.664290239 +0000 2024-03-14T21:14:20.100 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-14 21:14:03.015285296 +0000 2024-03-14T21:14:20.100 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-14T21:14:20.100 DEBUG:teuthology.orchestra.run.smithi163:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-14T21:14:20.201 INFO:tasks.workunit:timeout=6h 2024-03-14T21:14:20.201 INFO:tasks.workunit:cleanup=True 2024-03-14T21:14:20.201 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 234d532354068e06d01621fd032c3b663cead394 2024-03-14T21:14:20.258 INFO:tasks.workunit.client.0.smithi163.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-14T21:14:34.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:14:34.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:14:34.691 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:15:04.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:15:05.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:15:05.004 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:15:12.263 INFO:tasks.workunit.client.0.smithi163.stderr:Updating files: 67% (8139/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-03-14T21:15:12.898 INFO:tasks.workunit.client.0.smithi163.stderr:Note: switching to '234d532354068e06d01621fd032c3b663cead394'. 2024-03-14T21:15:12.898 INFO:tasks.workunit.client.0.smithi163.stderr: 2024-03-14T21:15:12.898 INFO:tasks.workunit.client.0.smithi163.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-14T21:15:12.898 INFO:tasks.workunit.client.0.smithi163.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr:state without impacting any branches by switching back to a branch. 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: git switch -c 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr:Or undo this operation with: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: git switch - 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr: 2024-03-14T21:15:12.899 INFO:tasks.workunit.client.0.smithi163.stderr:HEAD is now at 234d5323540 Merge branch 'wip-63405-reef' of https://github.com/rishabh-d-dave/ceph into wip-yuri4-testing-2024-03-13-0733-reef 2024-03-14T21:15:12.908 DEBUG:teuthology.orchestra.run.smithi163:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-14T21:15:12.961 INFO:tasks.workunit.client.0.smithi163.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-14T21:15:12.967 INFO:tasks.workunit.client.0.smithi163.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-14T21:15:12.967 INFO:tasks.workunit.client.0.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-14T21:15:13.863 INFO:tasks.workunit.client.0.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-14T21:15:13.973 INFO:tasks.workunit.client.0.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-14T21:15:14.057 INFO:tasks.workunit.client.0.smithi163.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-14T21:15:14.062 INFO:tasks.workunit.client.0.smithi163.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-14T21:15:14.062 INFO:tasks.workunit.client.0.smithi163.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-14T21:15:14.143 INFO:tasks.workunit.client.0.smithi163.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-14T21:15:14.152 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T21:15:14.152 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-14T21:15:14.199 INFO:tasks.workunit:Running workunits matching fs/fscrypt.sh on client.0... 2024-03-14T21:15:14.200 INFO:tasks.workunit:Running workunit fs/fscrypt.sh... 2024-03-14T21:15:14.201 DEBUG:teuthology.orchestra.run.smithi163:workunit test fs/fscrypt.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=234d532354068e06d01621fd032c3b663cead394 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh none dbench 2024-03-14T21:15:14.316 INFO:tasks.workunit.client.0.smithi163.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh 2024-03-14T21:15:14.319 INFO:tasks.workunit.client.0.smithi163.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-03-14T21:15:14.319 INFO:tasks.workunit.client.0.smithi163.stderr:+ '[' 2 -ne 2 ']' 2024-03-14T21:15:14.319 INFO:tasks.workunit.client.0.smithi163.stderr:+ fscrypt=none 2024-03-14T21:15:14.319 INFO:tasks.workunit.client.0.smithi163.stderr:+ testcase=dbench 2024-03-14T21:15:14.319 INFO:tasks.workunit.client.0.smithi163.stderr:+ testdir=fscrypt_test_none_dbench 2024-03-14T21:15:14.319 INFO:tasks.workunit.client.0.smithi163.stderr:+ mkdir fscrypt_test_none_dbench 2024-03-14T21:15:14.323 INFO:tasks.workunit.client.0.smithi163.stderr:+ XFSPROGS_DIR=xfprogs-dev-dir 2024-03-14T21:15:14.323 INFO:tasks.workunit.client.0.smithi163.stderr:+ XFSTESTS_DIR=xfstest-dev-dir 2024-03-14T21:15:14.325 INFO:tasks.workunit.client.0.smithi163.stderr:++ type -P xfs_io 2024-03-14T21:15:14.325 INFO:tasks.workunit.client.0.smithi163.stderr:+ export XFS_IO_PROG=/usr/sbin/xfs_io 2024-03-14T21:15:14.326 INFO:tasks.workunit.client.0.smithi163.stderr:+ XFS_IO_PROG=/usr/sbin/xfs_io 2024-03-14T21:15:14.326 INFO:tasks.workunit.client.0.smithi163.stderr:+ install_xfsprogs 2024-03-14T21:15:14.326 INFO:tasks.workunit.client.0.smithi163.stderr:+ local install_xfsprogs=0 2024-03-14T21:15:14.326 INFO:tasks.workunit.client.0.smithi163.stderr:+ xfs_io -c 'help add_enckey' 2024-03-14T21:15:14.327 INFO:tasks.workunit.client.0.smithi163.stderr:+ grep -q 'not found' 2024-03-14T21:15:14.334 INFO:tasks.workunit.client.0.smithi163.stderr:+ '[' 0 -eq 1 ']' 2024-03-14T21:15:14.334 INFO:tasks.workunit.client.0.smithi163.stderr:+ setup_xfstests_env 2024-03-14T21:15:14.334 INFO:tasks.workunit.client.0.smithi163.stderr:+ git clone https://git.ceph.com/xfstests-dev.git xfstest-dev-dir --depth 1 2024-03-14T21:15:14.339 INFO:tasks.workunit.client.0.smithi163.stderr:Cloning into 'xfstest-dev-dir'... 2024-03-14T21:15:19.993 INFO:tasks.workunit.client.0.smithi163.stderr:Updating files: 26% (1112/4261) Updating files: 27% (1151/4261) Updating files: 28% (1194/4261) Updating files: 29% (1236/4261) Updating files: 30% (1279/4261) Updating files: 31% (1321/4261) Updating files: 32% (1364/4261) Updating files: 33% (1407/4261) Updating files: 34% (1449/4261) Updating files: 35% (1492/4261) Updating files: 36% (1534/4261) Updating files: 37% (1577/4261) Updating files: 38% (1620/4261) Updating files: 39% (1662/4261) Updating files: 40% (1705/4261) Updating files: 41% (1748/4261) Updating files: 42% (1790/4261) Updating files: 43% (1833/4261) Updating files: 44% (1875/4261) Updating files: 45% (1918/4261) Updating files: 46% (1961/4261) Updating files: 47% (2003/4261) Updating files: 47% (2012/4261) Updating files: 48% (2046/4261) Updating files: 49% (2088/4261) Updating files: 50% (2131/4261) Updating files: 51% (2174/4261) Updating files: 52% (2216/4261) Updating files: 53% (2259/4261) Updating files: 54% (2301/4261) Updating files: 55% (2344/4261) Updating files: 56% (2387/4261) Updating files: 57% (2429/4261) Updating files: 58% (2472/4261) Updating files: 59% (2514/4261) Updating files: 60% (2557/4261) Updating files: 61% (2600/4261) Updating files: 62% (2642/4261) Updating files: 63% (2685/4261) Updating files: 64% (2728/4261) Updating files: 65% (2770/4261) Updating files: 66% (2813/4261) Updating files: 67% (2855/4261) Updating files: 68% (2898/4261) Updating files: 69% (2941/4261) Updating files: 70% (2983/4261) Updating files: 71% (3026/4261) Updating files: 72% (3068/4261) Updating files: 73% (3111/4261) Updating files: 74% (3154/4261) Updating files: 75% (3196/4261) Updating files: 76% (3239/4261) Updating files: 77% (3281/4261) Updating files: 78% (3324/4261) Updating files: 79% (3367/4261) Updating files: 80% (3409/4261) Updating files: 80% (3419/4261) Updating files: 81% (3452/4261) Updating files: 82% (3495/4261) Updating files: 83% (3537/4261) Updating files: 84% (3580/4261) Updating files: 85% (3622/4261) Updating files: 86% (3665/4261) Updating files: 87% (3708/4261) Updating files: 88% (3750/4261) Updating files: 89% (3793/4261) Updating files: 90% (3835/4261) Updating files: 91% (3878/4261) Updating files: 92% (3921/4261) Updating files: 93% (3963/4261) Updating files: 94% (4006/4261) Updating files: 95% (4048/4261) Updating files: 96% (4091/4261) Updating files: 97% (4134/4261) Updating files: 98% (4176/4261) Updating files: 99% (4219/4261) Updating files: 100% (4261/4261) Updating files: 100% (4261/4261), done. 2024-03-14T21:15:20.006 INFO:tasks.workunit.client.0.smithi163.stderr:+ pushd xfstest-dev-dir 2024-03-14T21:15:20.007 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp/xfstest-dev-dir ~/cephtest/mnt.0/client.0/tmp 2024-03-14T21:15:20.007 INFO:tasks.workunit.client.0.smithi163.stderr:+ . common/encrypt 2024-03-14T21:15:20.007 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY= 2024-03-14T21:15:20.007 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.009 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 1 2024-03-14T21:15:20.010 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x01' 2024-03-14T21:15:20.010 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.011 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 2 2024-03-14T21:15:20.011 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x02' 2024-03-14T21:15:20.011 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.012 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 3 2024-03-14T21:15:20.013 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x03' 2024-03-14T21:15:20.013 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.014 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 4 2024-03-14T21:15:20.015 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x04' 2024-03-14T21:15:20.015 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.016 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 5 2024-03-14T21:15:20.016 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x05' 2024-03-14T21:15:20.017 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.018 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 6 2024-03-14T21:15:20.018 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x06' 2024-03-14T21:15:20.018 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.020 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 7 2024-03-14T21:15:20.021 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x07' 2024-03-14T21:15:20.021 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.022 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 8 2024-03-14T21:15:20.022 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x08' 2024-03-14T21:15:20.022 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.023 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 9 2024-03-14T21:15:20.024 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x09' 2024-03-14T21:15:20.024 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.025 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 10 2024-03-14T21:15:20.025 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0a' 2024-03-14T21:15:20.026 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.027 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 11 2024-03-14T21:15:20.027 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0b' 2024-03-14T21:15:20.027 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.028 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 12 2024-03-14T21:15:20.029 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0c' 2024-03-14T21:15:20.029 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.030 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 13 2024-03-14T21:15:20.030 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0d' 2024-03-14T21:15:20.031 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.032 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 14 2024-03-14T21:15:20.032 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0e' 2024-03-14T21:15:20.032 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.033 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 15 2024-03-14T21:15:20.034 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0f' 2024-03-14T21:15:20.034 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.035 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 16 2024-03-14T21:15:20.036 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x10' 2024-03-14T21:15:20.036 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.037 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 17 2024-03-14T21:15:20.037 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x11' 2024-03-14T21:15:20.037 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.038 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 18 2024-03-14T21:15:20.039 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x12' 2024-03-14T21:15:20.039 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.040 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 19 2024-03-14T21:15:20.041 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x13' 2024-03-14T21:15:20.041 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.042 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 20 2024-03-14T21:15:20.042 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x14' 2024-03-14T21:15:20.042 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.043 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 21 2024-03-14T21:15:20.044 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x15' 2024-03-14T21:15:20.044 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.045 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 22 2024-03-14T21:15:20.045 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x16' 2024-03-14T21:15:20.045 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.046 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 23 2024-03-14T21:15:20.047 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x17' 2024-03-14T21:15:20.047 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.048 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 24 2024-03-14T21:15:20.049 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x18' 2024-03-14T21:15:20.049 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.049 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 25 2024-03-14T21:15:20.050 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x19' 2024-03-14T21:15:20.050 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.051 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 26 2024-03-14T21:15:20.052 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1a' 2024-03-14T21:15:20.052 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.053 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 27 2024-03-14T21:15:20.053 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1b' 2024-03-14T21:15:20.053 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.054 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 28 2024-03-14T21:15:20.055 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1c' 2024-03-14T21:15:20.055 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.056 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 29 2024-03-14T21:15:20.057 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1d' 2024-03-14T21:15:20.057 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.058 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 30 2024-03-14T21:15:20.058 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1e' 2024-03-14T21:15:20.058 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.059 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 31 2024-03-14T21:15:20.060 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1f' 2024-03-14T21:15:20.060 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.061 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 32 2024-03-14T21:15:20.061 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x20' 2024-03-14T21:15:20.061 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.062 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 33 2024-03-14T21:15:20.063 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x21' 2024-03-14T21:15:20.063 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.064 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 34 2024-03-14T21:15:20.064 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x22' 2024-03-14T21:15:20.064 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.065 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 35 2024-03-14T21:15:20.066 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x23' 2024-03-14T21:15:20.066 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.067 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 36 2024-03-14T21:15:20.067 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x24' 2024-03-14T21:15:20.068 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.069 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 37 2024-03-14T21:15:20.069 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x25' 2024-03-14T21:15:20.070 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.070 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 38 2024-03-14T21:15:20.071 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x26' 2024-03-14T21:15:20.071 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.072 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 39 2024-03-14T21:15:20.073 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x27' 2024-03-14T21:15:20.073 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.074 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 40 2024-03-14T21:15:20.074 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x28' 2024-03-14T21:15:20.074 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.075 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 41 2024-03-14T21:15:20.076 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x29' 2024-03-14T21:15:20.076 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.077 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 42 2024-03-14T21:15:20.078 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2a' 2024-03-14T21:15:20.078 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.082 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 43 2024-03-14T21:15:20.082 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2b' 2024-03-14T21:15:20.082 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.082 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 44 2024-03-14T21:15:20.082 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2c' 2024-03-14T21:15:20.082 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.082 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 45 2024-03-14T21:15:20.083 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2d' 2024-03-14T21:15:20.083 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.083 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 46 2024-03-14T21:15:20.084 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2e' 2024-03-14T21:15:20.084 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.085 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 47 2024-03-14T21:15:20.085 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2f' 2024-03-14T21:15:20.085 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.086 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 48 2024-03-14T21:15:20.087 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x30' 2024-03-14T21:15:20.087 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.088 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 49 2024-03-14T21:15:20.089 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x31' 2024-03-14T21:15:20.089 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.090 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 50 2024-03-14T21:15:20.090 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x32' 2024-03-14T21:15:20.090 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.091 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 51 2024-03-14T21:15:20.092 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x33' 2024-03-14T21:15:20.092 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.093 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 52 2024-03-14T21:15:20.094 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x34' 2024-03-14T21:15:20.094 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.094 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 53 2024-03-14T21:15:20.095 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x35' 2024-03-14T21:15:20.095 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.096 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 54 2024-03-14T21:15:20.097 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x36' 2024-03-14T21:15:20.097 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.098 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 55 2024-03-14T21:15:20.098 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x37' 2024-03-14T21:15:20.098 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.099 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 56 2024-03-14T21:15:20.100 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x38' 2024-03-14T21:15:20.100 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.101 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 57 2024-03-14T21:15:20.101 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x39' 2024-03-14T21:15:20.101 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.102 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 58 2024-03-14T21:15:20.103 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3a' 2024-03-14T21:15:20.103 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.104 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 59 2024-03-14T21:15:20.105 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3b' 2024-03-14T21:15:20.105 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.105 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 60 2024-03-14T21:15:20.106 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3c' 2024-03-14T21:15:20.106 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.107 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 61 2024-03-14T21:15:20.108 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3d' 2024-03-14T21:15:20.108 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.109 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 62 2024-03-14T21:15:20.109 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3e' 2024-03-14T21:15:20.109 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.110 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 63 2024-03-14T21:15:20.111 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3f' 2024-03-14T21:15:20.111 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:15:20.112 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 64 2024-03-14T21:15:20.113 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x40' 2024-03-14T21:15:20.113 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_KEY_DESCRIPTOR=0000111122223333 2024-03-14T21:15:20.113 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_KEY_IDENTIFIER=69b2f6edeee720cce0577937eb8a6751 2024-03-14T21:15:20.113 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_KEYRING_ID= 2024-03-14T21:15:20.114 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_256_XTS=1 2024-03-14T21:15:20.114 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_256_CTS=4 2024-03-14T21:15:20.114 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_128_CBC=5 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_128_CTS=6 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_ADIANTUM=9 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_256_HCTR2=10 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_POLICY_FLAG_DIRECT_KEY=0x04 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64=0x08 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32=0x10 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_KEY_SPEC_TYPE_DESCRIPTOR=1 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER=2 2024-03-14T21:15:20.115 INFO:tasks.workunit.client.0.smithi163.stderr:+ popd 2024-03-14T21:15:20.116 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-14T21:15:20.116 INFO:tasks.workunit.client.0.smithi163.stderr:++ _generate_raw_encryption_key 2024-03-14T21:15:20.116 INFO:tasks.workunit.client.0.smithi163.stderr:++ local raw= 2024-03-14T21:15:20.116 INFO:tasks.workunit.client.0.smithi163.stderr:++ local i 2024-03-14T21:15:20.117 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i = 0 )) 2024-03-14T21:15:20.117 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.117 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 98 2024-03-14T21:15:20.118 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62' 2024-03-14T21:15:20.118 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.118 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.119 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 80 2024-03-14T21:15:20.119 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50' 2024-03-14T21:15:20.120 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.120 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.120 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 194 2024-03-14T21:15:20.121 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2' 2024-03-14T21:15:20.121 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.121 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.122 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 27 2024-03-14T21:15:20.123 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b' 2024-03-14T21:15:20.123 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.123 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.124 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 26 2024-03-14T21:15:20.124 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a' 2024-03-14T21:15:20.124 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.125 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.125 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 168 2024-03-14T21:15:20.126 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8' 2024-03-14T21:15:20.126 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.126 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.127 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 76 2024-03-14T21:15:20.128 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c' 2024-03-14T21:15:20.128 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.128 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.129 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 87 2024-03-14T21:15:20.130 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57' 2024-03-14T21:15:20.130 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.130 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.130 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 127 2024-03-14T21:15:20.131 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f' 2024-03-14T21:15:20.131 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.131 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.132 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 238 2024-03-14T21:15:20.133 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee' 2024-03-14T21:15:20.133 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.133 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.134 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 106 2024-03-14T21:15:20.135 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a' 2024-03-14T21:15:20.135 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.135 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.136 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 64 2024-03-14T21:15:20.136 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40' 2024-03-14T21:15:20.137 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.137 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.137 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 113 2024-03-14T21:15:20.138 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71' 2024-03-14T21:15:20.138 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.138 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.139 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 137 2024-03-14T21:15:20.140 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89' 2024-03-14T21:15:20.140 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.140 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.140 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 104 2024-03-14T21:15:20.141 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68' 2024-03-14T21:15:20.141 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.141 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.142 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 159 2024-03-14T21:15:20.143 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f' 2024-03-14T21:15:20.143 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.143 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.144 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 0 2024-03-14T21:15:20.144 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00' 2024-03-14T21:15:20.144 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.145 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.145 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 85 2024-03-14T21:15:20.146 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55' 2024-03-14T21:15:20.146 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.147 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.147 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 182 2024-03-14T21:15:20.148 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6' 2024-03-14T21:15:20.148 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.148 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.149 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 126 2024-03-14T21:15:20.149 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e' 2024-03-14T21:15:20.149 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.149 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.150 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 125 2024-03-14T21:15:20.151 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d' 2024-03-14T21:15:20.151 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.151 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.152 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 128 2024-03-14T21:15:20.153 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80' 2024-03-14T21:15:20.153 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.153 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.154 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 225 2024-03-14T21:15:20.154 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1' 2024-03-14T21:15:20.154 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.154 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.155 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 81 2024-03-14T21:15:20.156 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51' 2024-03-14T21:15:20.156 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.156 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.157 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 93 2024-03-14T21:15:20.157 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d' 2024-03-14T21:15:20.157 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.157 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.158 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 228 2024-03-14T21:15:20.159 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4' 2024-03-14T21:15:20.159 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.159 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.160 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 87 2024-03-14T21:15:20.161 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57' 2024-03-14T21:15:20.161 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.161 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.162 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 86 2024-03-14T21:15:20.162 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56' 2024-03-14T21:15:20.162 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.162 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.163 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 71 2024-03-14T21:15:20.164 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47' 2024-03-14T21:15:20.164 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.164 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.165 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 229 2024-03-14T21:15:20.165 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5' 2024-03-14T21:15:20.165 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.165 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.166 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 15 2024-03-14T21:15:20.167 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f' 2024-03-14T21:15:20.167 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.167 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.168 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 243 2024-03-14T21:15:20.168 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3' 2024-03-14T21:15:20.169 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.169 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.169 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 237 2024-03-14T21:15:20.170 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed' 2024-03-14T21:15:20.170 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.170 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.171 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 139 2024-03-14T21:15:20.172 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b' 2024-03-14T21:15:20.172 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.172 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.172 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 126 2024-03-14T21:15:20.173 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e' 2024-03-14T21:15:20.173 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.173 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.174 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 68 2024-03-14T21:15:20.175 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44' 2024-03-14T21:15:20.175 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.175 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.176 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 109 2024-03-14T21:15:20.176 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d' 2024-03-14T21:15:20.177 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.177 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.177 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 236 2024-03-14T21:15:20.178 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec' 2024-03-14T21:15:20.178 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.178 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.179 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 236 2024-03-14T21:15:20.180 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec' 2024-03-14T21:15:20.180 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.180 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.181 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 14 2024-03-14T21:15:20.182 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e' 2024-03-14T21:15:20.182 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.182 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.182 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 110 2024-03-14T21:15:20.183 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e' 2024-03-14T21:15:20.183 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.183 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.184 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 244 2024-03-14T21:15:20.185 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4' 2024-03-14T21:15:20.185 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.185 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.186 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 252 2024-03-14T21:15:20.186 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc' 2024-03-14T21:15:20.187 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.187 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.187 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 255 2024-03-14T21:15:20.188 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff' 2024-03-14T21:15:20.188 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.188 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.189 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 109 2024-03-14T21:15:20.190 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d' 2024-03-14T21:15:20.190 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.190 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.191 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 33 2024-03-14T21:15:20.191 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21' 2024-03-14T21:15:20.192 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.192 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.192 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 24 2024-03-14T21:15:20.193 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18' 2024-03-14T21:15:20.193 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.193 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.194 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 121 2024-03-14T21:15:20.195 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79' 2024-03-14T21:15:20.195 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.195 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.196 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 140 2024-03-14T21:15:20.197 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c' 2024-03-14T21:15:20.197 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.197 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.197 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 224 2024-03-14T21:15:20.198 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0' 2024-03-14T21:15:20.198 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.198 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.199 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 241 2024-03-14T21:15:20.200 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1' 2024-03-14T21:15:20.200 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.200 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.201 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 30 2024-03-14T21:15:20.201 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e' 2024-03-14T21:15:20.201 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.202 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.203 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 34 2024-03-14T21:15:20.203 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22' 2024-03-14T21:15:20.203 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.203 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.204 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 31 2024-03-14T21:15:20.205 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f' 2024-03-14T21:15:20.205 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.205 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.206 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 245 2024-03-14T21:15:20.207 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5' 2024-03-14T21:15:20.207 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.207 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.208 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 228 2024-03-14T21:15:20.208 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4' 2024-03-14T21:15:20.208 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.208 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.209 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 194 2024-03-14T21:15:20.210 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2' 2024-03-14T21:15:20.210 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.210 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.211 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 41 2024-03-14T21:15:20.211 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29' 2024-03-14T21:15:20.212 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.212 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.212 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 252 2024-03-14T21:15:20.213 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc' 2024-03-14T21:15:20.213 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.213 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.214 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 193 2024-03-14T21:15:20.215 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1' 2024-03-14T21:15:20.215 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.215 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.216 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 236 2024-03-14T21:15:20.216 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec' 2024-03-14T21:15:20.216 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.216 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.217 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 34 2024-03-14T21:15:20.218 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22' 2024-03-14T21:15:20.218 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.218 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.219 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 157 2024-03-14T21:15:20.220 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22\x9d' 2024-03-14T21:15:20.220 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.220 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.221 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 137 2024-03-14T21:15:20.221 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22\x9d\x89' 2024-03-14T21:15:20.221 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:15:20.221 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:15:20.222 INFO:tasks.workunit.client.0.smithi163.stderr:++ echo '\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22\x9d\x89' 2024-03-14T21:15:20.222 INFO:tasks.workunit.client.0.smithi163.stderr:+ raw_key='\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22\x9d\x89' 2024-03-14T21:15:20.224 INFO:tasks.workunit.client.0.smithi163.stderr:++ _add_enckey fscrypt_test_none_dbench '\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22\x9d\x89' 2024-03-14T21:15:20.225 INFO:tasks.workunit.client.0.smithi163.stderr:++ local mnt=fscrypt_test_none_dbench 2024-03-14T21:15:20.225 INFO:tasks.workunit.client.0.smithi163.stderr:++ local 'raw_key=\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22\x9d\x89' 2024-03-14T21:15:20.225 INFO:tasks.workunit.client.0.smithi163.stderr:++ shift 2 2024-03-14T21:15:20.225 INFO:tasks.workunit.client.0.smithi163.stderr:++ awk '{print $NF}' 2024-03-14T21:15:20.226 INFO:tasks.workunit.client.0.smithi163.stderr:++ echo -ne '\x62\x50\xc2\x1b\x1a\xa8\x4c\x57\x7f\xee\x6a\x40\x71\x89\x68\x9f\x00\x55\xb6\x7e\x7d\x80\xe1\x51\x5d\xe4\x57\x56\x47\xe5\x0f\xf3\xed\x8b\x7e\x44\x6d\xec\xec\x0e\x6e\xf4\xfc\xff\x6d\x21\x18\x79\x8c\xe0\xf1\x1e\x22\x1f\xf5\xe4\xc2\x29\xfc\xc1\xec\x22\x9d\x89' 2024-03-14T21:15:20.226 INFO:tasks.workunit.client.0.smithi163.stderr:++ /usr/sbin/xfs_io -c 'add_enckey ' fscrypt_test_none_dbench 2024-03-14T21:15:20.244 INFO:tasks.workunit.client.0.smithi163.stderr:+ keyid=ecb087790eb395973abca5ea2341e896 2024-03-14T21:15:20.244 INFO:tasks.workunit.client.0.smithi163.stderr:+ case ${fscrypt} in 2024-03-14T21:15:20.244 INFO:tasks.workunit.client.0.smithi163.stderr:+ pushd fscrypt_test_none_dbench 2024-03-14T21:15:20.244 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp/fscrypt_test_none_dbench ~/cephtest/mnt.0/client.0/tmp 2024-03-14T21:15:20.245 INFO:tasks.workunit.client.0.smithi163.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/../suites/dbench.sh 2024-03-14T21:15:20.267 INFO:tasks.workunit.client.0.smithi163.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-14T21:15:20.267 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:15:20.267 INFO:tasks.workunit.client.0.smithi163.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-14T21:15:20.275 INFO:tasks.workunit.client.0.smithi163.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-14T21:15:20.375 INFO:tasks.workunit.client.0.smithi163.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-14T21:15:20.375 INFO:tasks.workunit.client.0.smithi163.stdout:releasing clients 2024-03-14T21:15:21.376 INFO:tasks.workunit.client.0.smithi163.stdout: 1 2088 26.64 MB/sec warmup 1 sec latency 8.628 ms 2024-03-14T21:15:22.376 INFO:tasks.workunit.client.0.smithi163.stdout: 1 4236 20.13 MB/sec warmup 2 sec latency 23.127 ms 2024-03-14T21:15:23.376 INFO:tasks.workunit.client.0.smithi163.stdout: 1 7240 19.29 MB/sec warmup 3 sec latency 8.195 ms 2024-03-14T21:15:24.376 INFO:tasks.workunit.client.0.smithi163.stdout: 1 9602 16.74 MB/sec warmup 4 sec latency 6.089 ms 2024-03-14T21:15:25.376 INFO:tasks.workunit.client.0.smithi163.stdout: 1 11861 16.08 MB/sec warmup 5 sec latency 21.192 ms 2024-03-14T21:15:26.376 INFO:tasks.workunit.client.0.smithi163.stdout: 1 14661 16.24 MB/sec warmup 6 sec latency 15.420 ms 2024-03-14T21:15:27.376 INFO:tasks.workunit.client.0.smithi163.stdout: 1 17684 16.40 MB/sec warmup 7 sec latency 6.216 ms 2024-03-14T21:15:28.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 20186 15.45 MB/sec warmup 8 sec latency 6.313 ms 2024-03-14T21:15:29.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 22317 15.18 MB/sec warmup 9 sec latency 16.965 ms 2024-03-14T21:15:30.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 25263 15.48 MB/sec warmup 10 sec latency 6.174 ms 2024-03-14T21:15:31.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 28080 15.29 MB/sec warmup 11 sec latency 16.041 ms 2024-03-14T21:15:32.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 30559 14.97 MB/sec warmup 12 sec latency 7.331 ms 2024-03-14T21:15:33.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 32848 14.91 MB/sec warmup 13 sec latency 5.853 ms 2024-03-14T21:15:34.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 35810 15.13 MB/sec warmup 14 sec latency 6.682 ms 2024-03-14T21:15:35.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:15:35.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:15:35.127 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:15:35.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38426 14.98 MB/sec warmup 15 sec latency 27.416 ms 2024-03-14T21:15:36.377 INFO:tasks.workunit.client.0.smithi163.stdout: 1 40587 14.78 MB/sec warmup 16 sec latency 12.986 ms 2024-03-14T21:15:37.378 INFO:tasks.workunit.client.0.smithi163.stdout: 1 43151 14.73 MB/sec warmup 17 sec latency 12.765 ms 2024-03-14T21:15:38.378 INFO:tasks.workunit.client.0.smithi163.stdout: 1 46228 14.89 MB/sec warmup 18 sec latency 6.605 ms 2024-03-14T21:15:39.378 INFO:tasks.workunit.client.0.smithi163.stdout: 1 48571 14.56 MB/sec warmup 19 sec latency 5.808 ms 2024-03-14T21:15:40.378 INFO:tasks.workunit.client.0.smithi163.stdout: 1 50601 14.47 MB/sec warmup 20 sec latency 19.195 ms 2024-03-14T21:15:41.378 INFO:tasks.workunit.client.0.smithi163.stdout: 1 53574 14.65 MB/sec warmup 21 sec latency 5.786 ms 2024-03-14T21:15:42.378 INFO:tasks.workunit.client.0.smithi163.stdout: 1 56219 14.56 MB/sec warmup 22 sec latency 9.492 ms 2024-03-14T21:15:43.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 58653 14.45 MB/sec warmup 23 sec latency 16.110 ms 2024-03-14T21:15:44.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 61129 14.44 MB/sec warmup 24 sec latency 6.531 ms 2024-03-14T21:15:45.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 64115 14.59 MB/sec warmup 25 sec latency 37.238 ms 2024-03-14T21:15:46.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 66744 14.48 MB/sec warmup 26 sec latency 20.465 ms 2024-03-14T21:15:47.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 68805 14.42 MB/sec warmup 27 sec latency 5.828 ms 2024-03-14T21:15:48.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 71442 14.41 MB/sec warmup 28 sec latency 14.609 ms 2024-03-14T21:15:49.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 74613 14.52 MB/sec warmup 29 sec latency 5.921 ms 2024-03-14T21:15:50.379 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77011 14.34 MB/sec warmup 30 sec latency 6.112 ms 2024-03-14T21:15:51.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77307 13.99 MB/sec warmup 31 sec latency 453.466 ms 2024-03-14T21:15:52.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77307 13.55 MB/sec warmup 32 sec latency 1415.317 ms 2024-03-14T21:15:53.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 78281 13.36 MB/sec warmup 33 sec latency 1884.997 ms 2024-03-14T21:15:54.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 80666 13.23 MB/sec warmup 34 sec latency 6.926 ms 2024-03-14T21:15:55.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 82857 13.23 MB/sec warmup 35 sec latency 5.796 ms 2024-03-14T21:15:56.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 85426 13.32 MB/sec warmup 36 sec latency 98.816 ms 2024-03-14T21:15:57.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 87730 13.19 MB/sec warmup 37 sec latency 37.870 ms 2024-03-14T21:15:58.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89985 13.19 MB/sec warmup 38 sec latency 8.776 ms 2024-03-14T21:15:59.380 INFO:tasks.workunit.client.0.smithi163.stdout: 1 92732 13.29 MB/sec warmup 39 sec latency 20.133 ms 2024-03-14T21:16:00.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 95767 13.39 MB/sec warmup 40 sec latency 7.097 ms 2024-03-14T21:16:01.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 98252 13.28 MB/sec warmup 41 sec latency 5.594 ms 2024-03-14T21:16:02.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 99964 13.25 MB/sec warmup 42 sec latency 51.611 ms 2024-03-14T21:16:03.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 103050 13.35 MB/sec warmup 43 sec latency 6.279 ms 2024-03-14T21:16:04.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 105512 13.26 MB/sec warmup 44 sec latency 5.928 ms 2024-03-14T21:16:05.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:16:05.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:16:05.256 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:16:05.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 107518 13.26 MB/sec warmup 45 sec latency 75.757 ms 2024-03-14T21:16:06.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 110368 13.34 MB/sec warmup 46 sec latency 89.644 ms 2024-03-14T21:16:07.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 112468 13.22 MB/sec warmup 47 sec latency 118.823 ms 2024-03-14T21:16:08.381 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114776 13.23 MB/sec warmup 48 sec latency 5.627 ms 2024-03-14T21:16:09.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 117625 13.31 MB/sec warmup 49 sec latency 6.423 ms 2024-03-14T21:16:10.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 120467 13.33 MB/sec warmup 50 sec latency 14.955 ms 2024-03-14T21:16:11.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 123001 13.30 MB/sec warmup 51 sec latency 18.969 ms 2024-03-14T21:16:12.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 124739 13.27 MB/sec warmup 52 sec latency 44.046 ms 2024-03-14T21:16:13.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 127812 13.36 MB/sec warmup 53 sec latency 6.052 ms 2024-03-14T21:16:14.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 130169 13.27 MB/sec warmup 54 sec latency 7.038 ms 2024-03-14T21:16:15.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 132453 13.28 MB/sec warmup 55 sec latency 21.690 ms 2024-03-14T21:16:16.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 135276 13.35 MB/sec warmup 56 sec latency 5.445 ms 2024-03-14T21:16:17.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 137871 13.34 MB/sec warmup 57 sec latency 66.841 ms 2024-03-14T21:16:18.382 INFO:tasks.workunit.client.0.smithi163.stdout: 1 140232 13.31 MB/sec warmup 58 sec latency 18.541 ms 2024-03-14T21:16:19.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 142776 13.33 MB/sec warmup 59 sec latency 6.170 ms 2024-03-14T21:16:20.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 145735 13.41 MB/sec warmup 60 sec latency 11.318 ms 2024-03-14T21:16:21.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 148374 13.40 MB/sec warmup 61 sec latency 6.016 ms 2024-03-14T21:16:22.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 150425 13.37 MB/sec warmup 62 sec latency 38.353 ms 2024-03-14T21:16:23.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 153013 13.38 MB/sec warmup 63 sec latency 24.744 ms 2024-03-14T21:16:24.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 155816 13.40 MB/sec warmup 64 sec latency 21.594 ms 2024-03-14T21:16:25.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 158354 13.36 MB/sec warmup 65 sec latency 5.764 ms 2024-03-14T21:16:26.383 INFO:tasks.workunit.client.0.smithi163.stdout: 1 160466 13.36 MB/sec warmup 66 sec latency 17.676 ms 2024-03-14T21:16:27.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 163455 13.44 MB/sec warmup 67 sec latency 16.620 ms 2024-03-14T21:16:28.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 165776 13.36 MB/sec warmup 68 sec latency 75.584 ms 2024-03-14T21:16:29.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 168142 13.40 MB/sec warmup 69 sec latency 6.164 ms 2024-03-14T21:16:30.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170858 13.41 MB/sec warmup 70 sec latency 7.230 ms 2024-03-14T21:16:31.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 173892 13.47 MB/sec warmup 71 sec latency 21.708 ms 2024-03-14T21:16:32.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 176252 13.40 MB/sec warmup 72 sec latency 8.585 ms 2024-03-14T21:16:33.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 178189 13.39 MB/sec warmup 73 sec latency 28.094 ms 2024-03-14T21:16:34.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 181173 13.46 MB/sec warmup 74 sec latency 6.652 ms 2024-03-14T21:16:35.384 INFO:tasks.workunit.client.0.smithi163.stdout: 1 183800 13.44 MB/sec warmup 75 sec latency 6.630 ms 2024-03-14T21:16:35.567 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:16:35.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:16:35.573 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:16:36.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 185688 13.39 MB/sec warmup 76 sec latency 21.441 ms 2024-03-14T21:16:37.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 188444 13.43 MB/sec warmup 77 sec latency 13.337 ms 2024-03-14T21:16:38.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 190878 13.41 MB/sec warmup 78 sec latency 100.909 ms 2024-03-14T21:16:39.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 192950 13.40 MB/sec warmup 79 sec latency 6.370 ms 2024-03-14T21:16:40.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195532 13.41 MB/sec warmup 80 sec latency 6.021 ms 2024-03-14T21:16:41.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 198070 13.40 MB/sec warmup 81 sec latency 6.625 ms 2024-03-14T21:16:42.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 200158 13.38 MB/sec warmup 82 sec latency 20.934 ms 2024-03-14T21:16:43.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 202569 13.38 MB/sec warmup 83 sec latency 70.001 ms 2024-03-14T21:16:44.385 INFO:tasks.workunit.client.0.smithi163.stdout: 1 205394 13.38 MB/sec warmup 84 sec latency 5.655 ms 2024-03-14T21:16:45.386 INFO:tasks.workunit.client.0.smithi163.stdout: 1 208067 13.37 MB/sec warmup 85 sec latency 5.613 ms 2024-03-14T21:16:46.386 INFO:tasks.workunit.client.0.smithi163.stdout: 1 210165 13.37 MB/sec warmup 86 sec latency 38.274 ms 2024-03-14T21:16:47.386 INFO:tasks.workunit.client.0.smithi163.stdout: 1 213178 13.43 MB/sec warmup 87 sec latency 5.543 ms 2024-03-14T21:16:48.387 INFO:tasks.workunit.client.0.smithi163.stdout: 1 215342 13.36 MB/sec warmup 88 sec latency 25.667 ms 2024-03-14T21:16:49.386 INFO:tasks.workunit.client.0.smithi163.stdout: 1 217673 13.37 MB/sec warmup 89 sec latency 5.368 ms 2024-03-14T21:16:50.386 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220503 13.41 MB/sec warmup 90 sec latency 6.072 ms 2024-03-14T21:16:51.387 INFO:tasks.workunit.client.0.smithi163.stdout: 1 223573 13.45 MB/sec warmup 91 sec latency 26.187 ms 2024-03-14T21:16:52.387 INFO:tasks.workunit.client.0.smithi163.stdout: 1 225909 13.40 MB/sec warmup 92 sec latency 25.110 ms 2024-03-14T21:16:53.387 INFO:tasks.workunit.client.0.smithi163.stdout: 1 227979 13.39 MB/sec warmup 93 sec latency 18.058 ms 2024-03-14T21:16:54.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 230623 13.43 MB/sec warmup 94 sec latency 58.645 ms 2024-03-14T21:16:55.387 INFO:tasks.workunit.client.0.smithi163.stdout: 1 233090 13.39 MB/sec warmup 95 sec latency 5.892 ms 2024-03-14T21:16:56.387 INFO:tasks.workunit.client.0.smithi163.stdout: 1 235331 13.39 MB/sec warmup 96 sec latency 37.045 ms 2024-03-14T21:16:57.387 INFO:tasks.workunit.client.0.smithi163.stdout: 1 238112 13.42 MB/sec warmup 97 sec latency 16.700 ms 2024-03-14T21:16:58.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 241051 13.43 MB/sec warmup 98 sec latency 5.630 ms 2024-03-14T21:16:59.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 242648 13.40 MB/sec warmup 99 sec latency 70.471 ms 2024-03-14T21:17:00.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 245211 13.40 MB/sec warmup 100 sec latency 50.070 ms 2024-03-14T21:17:01.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 248119 13.41 MB/sec warmup 101 sec latency 6.117 ms 2024-03-14T21:17:02.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 250578 13.39 MB/sec warmup 102 sec latency 6.308 ms 2024-03-14T21:17:03.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 252803 13.39 MB/sec warmup 103 sec latency 24.556 ms 2024-03-14T21:17:04.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 255632 13.44 MB/sec warmup 104 sec latency 61.732 ms 2024-03-14T21:17:05.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 258137 13.42 MB/sec warmup 105 sec latency 25.077 ms 2024-03-14T21:17:05.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:17:05.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:17:05.705 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:17:06.388 INFO:tasks.workunit.client.0.smithi163.stdout: 1 260208 13.39 MB/sec warmup 106 sec latency 7.342 ms 2024-03-14T21:17:07.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 262957 13.42 MB/sec warmup 107 sec latency 13.893 ms 2024-03-14T21:17:08.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 266054 13.46 MB/sec warmup 108 sec latency 5.851 ms 2024-03-14T21:17:09.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 268396 13.41 MB/sec warmup 109 sec latency 59.458 ms 2024-03-14T21:17:10.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270429 13.41 MB/sec warmup 110 sec latency 78.450 ms 2024-03-14T21:17:11.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 273399 13.45 MB/sec warmup 111 sec latency 20.983 ms 2024-03-14T21:17:12.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 276044 13.44 MB/sec warmup 112 sec latency 6.009 ms 2024-03-14T21:17:13.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 278397 13.43 MB/sec warmup 113 sec latency 5.913 ms 2024-03-14T21:17:14.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 280919 13.44 MB/sec warmup 114 sec latency 14.015 ms 2024-03-14T21:17:15.389 INFO:tasks.workunit.client.0.smithi163.stdout: 1 283543 13.44 MB/sec warmup 115 sec latency 63.558 ms 2024-03-14T21:17:16.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 285877 13.42 MB/sec warmup 116 sec latency 10.833 ms 2024-03-14T21:17:17.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 288269 13.42 MB/sec warmup 117 sec latency 9.325 ms 2024-03-14T21:17:18.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 291263 13.46 MB/sec warmup 118 sec latency 6.112 ms 2024-03-14T21:17:19.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 294224 13.47 MB/sec warmup 119 sec latency 10.776 ms 2024-03-14T21:17:21.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298496 14.05 MB/sec execute 1 sec latency 74.999 ms 2024-03-14T21:17:22.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 301582 15.84 MB/sec execute 2 sec latency 6.297 ms 2024-03-14T21:17:23.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 303990 13.61 MB/sec execute 3 sec latency 6.324 ms 2024-03-14T21:17:24.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 306078 13.34 MB/sec execute 4 sec latency 7.133 ms 2024-03-14T21:17:25.390 INFO:tasks.workunit.client.0.smithi163.stdout: 1 308755 14.02 MB/sec execute 5 sec latency 33.108 ms 2024-03-14T21:17:26.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 310949 13.05 MB/sec execute 6 sec latency 66.265 ms 2024-03-14T21:17:27.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 313068 13.03 MB/sec execute 7 sec latency 8.665 ms 2024-03-14T21:17:28.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 315996 13.66 MB/sec execute 8 sec latency 6.219 ms 2024-03-14T21:17:29.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 318708 13.58 MB/sec execute 9 sec latency 6.065 ms 2024-03-14T21:17:30.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 320508 13.09 MB/sec execute 10 sec latency 46.308 ms 2024-03-14T21:17:31.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323218 13.44 MB/sec execute 11 sec latency 32.228 ms 2024-03-14T21:17:32.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 326169 13.53 MB/sec execute 12 sec latency 5.868 ms 2024-03-14T21:17:33.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 328733 13.38 MB/sec execute 13 sec latency 23.197 ms 2024-03-14T21:17:34.391 INFO:tasks.workunit.client.0.smithi163.stdout: 1 331049 13.44 MB/sec execute 14 sec latency 5.693 ms 2024-03-14T21:17:35.392 INFO:tasks.workunit.client.0.smithi163.stdout: 1 333750 13.66 MB/sec execute 15 sec latency 79.572 ms 2024-03-14T21:17:35.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:17:35.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:17:35.924 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:17:36.392 INFO:tasks.workunit.client.0.smithi163.stdout: 1 336392 13.61 MB/sec execute 16 sec latency 25.455 ms 2024-03-14T21:17:37.392 INFO:tasks.workunit.client.0.smithi163.stdout: 1 338673 13.51 MB/sec execute 17 sec latency 5.459 ms 2024-03-14T21:17:38.392 INFO:tasks.workunit.client.0.smithi163.stdout: 1 341251 13.54 MB/sec execute 18 sec latency 9.786 ms 2024-03-14T21:17:39.392 INFO:tasks.workunit.client.0.smithi163.stdout: 1 344300 13.80 MB/sec execute 19 sec latency 20.159 ms 2024-03-14T21:17:40.392 INFO:tasks.workunit.client.0.smithi163.stdout: 1 346469 13.48 MB/sec execute 20 sec latency 66.608 ms 2024-03-14T21:17:41.392 INFO:tasks.workunit.client.0.smithi163.stdout: 1 348523 13.45 MB/sec execute 21 sec latency 45.222 ms 2024-03-14T21:17:42.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 351475 13.66 MB/sec execute 22 sec latency 6.578 ms 2024-03-14T21:17:43.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 354145 13.62 MB/sec execute 23 sec latency 6.355 ms 2024-03-14T21:17:44.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 356350 13.55 MB/sec execute 24 sec latency 8.447 ms 2024-03-14T21:17:45.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 358960 13.57 MB/sec execute 25 sec latency 5.759 ms 2024-03-14T21:17:46.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 361604 13.60 MB/sec execute 26 sec latency 95.643 ms 2024-03-14T21:17:47.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 363829 13.47 MB/sec execute 27 sec latency 26.776 ms 2024-03-14T21:17:48.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 366149 13.49 MB/sec execute 28 sec latency 5.905 ms 2024-03-14T21:17:49.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 369146 13.66 MB/sec execute 29 sec latency 9.756 ms 2024-03-14T21:17:50.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 371777 13.61 MB/sec execute 30 sec latency 7.259 ms 2024-03-14T21:17:51.393 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373816 13.57 MB/sec execute 31 sec latency 60.011 ms 2024-03-14T21:17:52.394 INFO:tasks.workunit.client.0.smithi163.stdout: 1 376373 13.58 MB/sec execute 32 sec latency 72.927 ms 2024-03-14T21:17:53.394 INFO:tasks.workunit.client.0.smithi163.stdout: 1 379297 13.61 MB/sec execute 33 sec latency 6.044 ms 2024-03-14T21:17:54.394 INFO:tasks.workunit.client.0.smithi163.stdout: 1 381817 13.52 MB/sec execute 34 sec latency 8.278 ms 2024-03-14T21:17:55.394 INFO:tasks.workunit.client.0.smithi163.stdout: 1 384028 13.54 MB/sec execute 35 sec latency 6.661 ms 2024-03-14T21:17:56.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 387014 13.67 MB/sec execute 36 sec latency 16.230 ms 2024-03-14T21:17:57.394 INFO:tasks.workunit.client.0.smithi163.stdout: 1 389246 13.52 MB/sec execute 37 sec latency 73.619 ms 2024-03-14T21:17:58.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 391442 13.51 MB/sec execute 38 sec latency 27.671 ms 2024-03-14T21:17:59.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 394239 13.60 MB/sec execute 39 sec latency 5.723 ms 2024-03-14T21:18:00.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 397278 13.69 MB/sec execute 40 sec latency 26.673 ms 2024-03-14T21:18:01.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 399751 13.58 MB/sec execute 41 sec latency 6.160 ms 2024-03-14T21:18:02.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 401614 13.55 MB/sec execute 42 sec latency 69.631 ms 2024-03-14T21:18:03.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 404629 13.66 MB/sec execute 43 sec latency 23.903 ms 2024-03-14T21:18:04.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 407022 13.54 MB/sec execute 44 sec latency 6.072 ms 2024-03-14T21:18:05.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 409455 13.60 MB/sec execute 45 sec latency 10.874 ms 2024-03-14T21:18:06.064 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:18:06.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:18:06.069 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:18:06.395 INFO:tasks.workunit.client.0.smithi163.stdout: 1 411924 13.61 MB/sec execute 46 sec latency 51.848 ms 2024-03-14T21:18:07.396 INFO:tasks.workunit.client.0.smithi163.stdout: 1 414259 13.55 MB/sec execute 47 sec latency 96.167 ms 2024-03-14T21:18:08.396 INFO:tasks.workunit.client.0.smithi163.stdout: 1 416343 13.49 MB/sec execute 48 sec latency 14.711 ms 2024-03-14T21:18:09.396 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419048 13.55 MB/sec execute 49 sec latency 6.345 ms 2024-03-14T21:18:10.396 INFO:tasks.workunit.client.0.smithi163.stdout: 1 422106 13.63 MB/sec execute 50 sec latency 7.774 ms 2024-03-14T21:18:11.396 INFO:tasks.workunit.client.0.smithi163.stdout: 1 424510 13.53 MB/sec execute 51 sec latency 14.581 ms 2024-03-14T21:18:12.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 426350 13.51 MB/sec execute 52 sec latency 30.608 ms 2024-03-14T21:18:13.396 INFO:tasks.workunit.client.0.smithi163.stdout: 1 429303 13.59 MB/sec execute 53 sec latency 17.411 ms 2024-03-14T21:18:14.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 431719 13.50 MB/sec execute 54 sec latency 6.213 ms 2024-03-14T21:18:15.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 434191 13.56 MB/sec execute 55 sec latency 7.584 ms 2024-03-14T21:18:16.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 436865 13.57 MB/sec execute 56 sec latency 28.716 ms 2024-03-14T21:18:17.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 439978 13.64 MB/sec execute 57 sec latency 9.612 ms 2024-03-14T21:18:18.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 441831 13.52 MB/sec execute 58 sec latency 54.521 ms 2024-03-14T21:18:19.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 444317 13.54 MB/sec execute 59 sec latency 6.149 ms 2024-03-14T21:18:20.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 447241 13.61 MB/sec execute 60 sec latency 7.725 ms 2024-03-14T21:18:21.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 449742 13.58 MB/sec execute 61 sec latency 17.960 ms 2024-03-14T21:18:22.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 451751 13.52 MB/sec execute 62 sec latency 9.081 ms 2024-03-14T21:18:23.397 INFO:tasks.workunit.client.0.smithi163.stdout: 1 454368 13.57 MB/sec execute 63 sec latency 73.899 ms 2024-03-14T21:18:24.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 457017 13.56 MB/sec execute 64 sec latency 17.970 ms 2024-03-14T21:18:25.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 458725 13.67 MB/sec execute 65 sec latency 48.465 ms 2024-03-14T21:18:26.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 461649 13.79 MB/sec execute 66 sec latency 25.977 ms 2024-03-14T21:18:27.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 463422 13.73 MB/sec execute 67 sec latency 22.041 ms 2024-03-14T21:18:28.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 465748 13.73 MB/sec execute 68 sec latency 116.562 ms 2024-03-14T21:18:29.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 468240 13.69 MB/sec execute 69 sec latency 21.544 ms 2024-03-14T21:18:30.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 470446 13.69 MB/sec execute 70 sec latency 6.482 ms 2024-03-14T21:18:31.398 INFO:tasks.workunit.client.0.smithi163.stdout: 1 473100 13.69 MB/sec execute 71 sec latency 7.315 ms 2024-03-14T21:18:32.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 476191 13.75 MB/sec execute 72 sec latency 19.293 ms 2024-03-14T21:18:33.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 478331 13.66 MB/sec execute 73 sec latency 62.763 ms 2024-03-14T21:18:34.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 480530 13.66 MB/sec execute 74 sec latency 23.169 ms 2024-03-14T21:18:35.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 483464 13.72 MB/sec execute 75 sec latency 6.067 ms 2024-03-14T21:18:36.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:18:36.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:18:36.351 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:18:36.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 485984 13.69 MB/sec execute 76 sec latency 6.383 ms 2024-03-14T21:18:37.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 488267 13.68 MB/sec execute 77 sec latency 15.277 ms 2024-03-14T21:18:38.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 490749 13.69 MB/sec execute 78 sec latency 27.482 ms 2024-03-14T21:18:39.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 492994 13.64 MB/sec execute 79 sec latency 55.926 ms 2024-03-14T21:18:40.399 INFO:tasks.workunit.client.0.smithi163.stdout: 1 495305 13.65 MB/sec execute 80 sec latency 6.094 ms 2024-03-14T21:18:41.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 497976 13.65 MB/sec execute 81 sec latency 5.828 ms 2024-03-14T21:18:42.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 501054 13.70 MB/sec execute 82 sec latency 7.719 ms 2024-03-14T21:18:43.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 503432 13.65 MB/sec execute 83 sec latency 15.181 ms 2024-03-14T21:18:44.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 505314 13.63 MB/sec execute 84 sec latency 67.039 ms 2024-03-14T21:18:45.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 508264 13.68 MB/sec execute 85 sec latency 23.254 ms 2024-03-14T21:18:46.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 510796 13.66 MB/sec execute 86 sec latency 12.927 ms 2024-03-14T21:18:47.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 512860 13.62 MB/sec execute 87 sec latency 71.775 ms 2024-03-14T21:18:48.400 INFO:tasks.workunit.client.0.smithi163.stdout: 1 515651 13.65 MB/sec execute 88 sec latency 7.185 ms 2024-03-14T21:18:49.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 518135 13.64 MB/sec execute 89 sec latency 81.416 ms 2024-03-14T21:18:50.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 520322 13.62 MB/sec execute 90 sec latency 31.528 ms 2024-03-14T21:18:51.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 522815 13.62 MB/sec execute 91 sec latency 7.519 ms 2024-03-14T21:18:52.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 525571 13.63 MB/sec execute 92 sec latency 6.619 ms 2024-03-14T21:18:53.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 527960 13.60 MB/sec execute 93 sec latency 26.899 ms 2024-03-14T21:18:54.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 529909 13.60 MB/sec execute 94 sec latency 53.656 ms 2024-03-14T21:18:55.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 532732 13.61 MB/sec execute 95 sec latency 56.693 ms 2024-03-14T21:18:56.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 535136 13.57 MB/sec execute 96 sec latency 6.511 ms 2024-03-14T21:18:57.401 INFO:tasks.workunit.client.0.smithi163.stdout: 1 537365 13.58 MB/sec execute 97 sec latency 6.232 ms 2024-03-14T21:18:58.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 540302 13.62 MB/sec execute 98 sec latency 7.336 ms 2024-03-14T21:18:59.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 542878 13.61 MB/sec execute 99 sec latency 10.534 ms 2024-03-14T21:19:00.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 544347 13.55 MB/sec execute 100 sec latency 83.407 ms 2024-03-14T21:19:01.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 547307 13.60 MB/sec execute 101 sec latency 5.825 ms 2024-03-14T21:19:02.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 549646 13.55 MB/sec execute 102 sec latency 12.494 ms 2024-03-14T21:19:03.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 551792 13.54 MB/sec execute 103 sec latency 6.982 ms 2024-03-14T21:19:04.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 554480 13.57 MB/sec execute 104 sec latency 14.806 ms 2024-03-14T21:19:05.402 INFO:tasks.workunit.client.0.smithi163.stdout: 1 556590 13.52 MB/sec execute 105 sec latency 53.117 ms 2024-03-14T21:19:06.403 INFO:tasks.workunit.client.0.smithi163.stdout: 1 558642 13.51 MB/sec execute 106 sec latency 21.943 ms 2024-03-14T21:19:06.482 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:19:06.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:19:06.487 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:19:07.403 INFO:tasks.workunit.client.0.smithi163.stdout: 1 561576 13.55 MB/sec execute 107 sec latency 6.746 ms 2024-03-14T21:19:08.403 INFO:tasks.workunit.client.0.smithi163.stdout: 1 564173 13.54 MB/sec execute 108 sec latency 5.838 ms 2024-03-14T21:19:09.403 INFO:tasks.workunit.client.0.smithi163.stdout: 1 566303 13.53 MB/sec execute 109 sec latency 16.372 ms 2024-03-14T21:19:10.403 INFO:tasks.workunit.client.0.smithi163.stdout: 1 568698 13.53 MB/sec execute 110 sec latency 41.706 ms 2024-03-14T21:19:11.403 INFO:tasks.workunit.client.0.smithi163.stdout: 1 571318 13.52 MB/sec execute 111 sec latency 17.252 ms 2024-03-14T21:19:12.403 INFO:tasks.workunit.client.0.smithi163.stdout: 1 573514 13.51 MB/sec execute 112 sec latency 6.575 ms 2024-03-14T21:19:13.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 576101 13.51 MB/sec execute 113 sec latency 5.397 ms 2024-03-14T21:19:14.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 579124 13.55 MB/sec execute 114 sec latency 6.754 ms 2024-03-14T21:19:15.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 580956 13.49 MB/sec execute 115 sec latency 87.269 ms 2024-03-14T21:19:16.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 583249 13.49 MB/sec execute 116 sec latency 29.555 ms 2024-03-14T21:19:17.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 586291 13.54 MB/sec execute 117 sec latency 6.630 ms 2024-03-14T21:19:18.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 588642 13.49 MB/sec execute 118 sec latency 17.460 ms 2024-03-14T21:19:19.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 590801 13.49 MB/sec execute 119 sec latency 6.175 ms 2024-03-14T21:19:20.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 593537 13.52 MB/sec execute 120 sec latency 20.664 ms 2024-03-14T21:19:21.404 INFO:tasks.workunit.client.0.smithi163.stdout: 1 595766 13.47 MB/sec execute 121 sec latency 66.138 ms 2024-03-14T21:19:22.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 597962 13.47 MB/sec execute 122 sec latency 6.448 ms 2024-03-14T21:19:23.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 600703 13.50 MB/sec execute 123 sec latency 6.194 ms 2024-03-14T21:19:24.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 603721 13.53 MB/sec execute 124 sec latency 19.676 ms 2024-03-14T21:19:25.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 606194 13.49 MB/sec execute 125 sec latency 6.690 ms 2024-03-14T21:19:26.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 607913 13.48 MB/sec execute 126 sec latency 93.454 ms 2024-03-14T21:19:27.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 610898 13.51 MB/sec execute 127 sec latency 8.028 ms 2024-03-14T21:19:28.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 613240 13.48 MB/sec execute 128 sec latency 21.511 ms 2024-03-14T21:19:29.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 615191 13.47 MB/sec execute 129 sec latency 16.282 ms 2024-03-14T21:19:30.405 INFO:tasks.workunit.client.0.smithi163.stdout: 1 618193 13.51 MB/sec execute 130 sec latency 6.548 ms 2024-03-14T21:19:31.406 INFO:tasks.workunit.client.0.smithi163.stdout: 1 619844 13.45 MB/sec execute 131 sec latency 107.701 ms 2024-03-14T21:19:32.406 INFO:tasks.workunit.client.0.smithi163.stdout: 1 622249 13.45 MB/sec execute 132 sec latency 8.568 ms 2024-03-14T21:19:33.406 INFO:tasks.workunit.client.0.smithi163.stdout: 1 625288 13.49 MB/sec execute 133 sec latency 6.999 ms 2024-03-14T21:19:34.406 INFO:tasks.workunit.client.0.smithi163.stdout: 1 627657 13.45 MB/sec execute 134 sec latency 9.560 ms 2024-03-14T21:19:35.406 INFO:tasks.workunit.client.0.smithi163.stdout: 1 630027 13.47 MB/sec execute 135 sec latency 6.174 ms 2024-03-14T21:19:36.406 INFO:tasks.workunit.client.0.smithi163.stdout: 1 632460 13.47 MB/sec execute 136 sec latency 89.553 ms 2024-03-14T21:19:36.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:19:36.604 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:19:36.607 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:19:37.407 INFO:tasks.workunit.client.0.smithi163.stdout: 1 634462 13.43 MB/sec execute 137 sec latency 220.771 ms 2024-03-14T21:19:38.407 INFO:tasks.workunit.client.0.smithi163.stdout: 1 636593 13.42 MB/sec execute 138 sec latency 6.106 ms 2024-03-14T21:19:39.407 INFO:tasks.workunit.client.0.smithi163.stdout: 1 639525 13.46 MB/sec execute 139 sec latency 9.361 ms 2024-03-14T21:19:40.407 INFO:tasks.workunit.client.0.smithi163.stdout: 1 642015 13.44 MB/sec execute 140 sec latency 31.203 ms 2024-03-14T21:19:41.407 INFO:tasks.workunit.client.0.smithi163.stdout: 1 644185 13.44 MB/sec execute 141 sec latency 22.912 ms 2024-03-14T21:19:42.407 INFO:tasks.workunit.client.0.smithi163.stdout: 1 646656 13.44 MB/sec execute 142 sec latency 49.553 ms 2024-03-14T21:19:43.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 649226 13.43 MB/sec execute 143 sec latency 6.134 ms 2024-03-14T21:19:44.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 651122 13.40 MB/sec execute 144 sec latency 8.632 ms 2024-03-14T21:19:45.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 653865 13.43 MB/sec execute 145 sec latency 8.872 ms 2024-03-14T21:19:46.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 656625 13.43 MB/sec execute 146 sec latency 27.021 ms 2024-03-14T21:19:47.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 658167 13.39 MB/sec execute 147 sec latency 34.649 ms 2024-03-14T21:19:48.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 660913 13.42 MB/sec execute 148 sec latency 6.654 ms 2024-03-14T21:19:49.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 663638 13.41 MB/sec execute 149 sec latency 6.379 ms 2024-03-14T21:19:50.408 INFO:tasks.workunit.client.0.smithi163.stdout: 1 665713 13.40 MB/sec execute 150 sec latency 30.482 ms 2024-03-14T21:19:51.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 668175 13.40 MB/sec execute 151 sec latency 6.693 ms 2024-03-14T21:19:52.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 670537 13.40 MB/sec execute 152 sec latency 57.248 ms 2024-03-14T21:19:53.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 672813 13.39 MB/sec execute 153 sec latency 6.404 ms 2024-03-14T21:19:54.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 675348 13.39 MB/sec execute 154 sec latency 6.935 ms 2024-03-14T21:19:55.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 678398 13.42 MB/sec execute 155 sec latency 10.575 ms 2024-03-14T21:19:56.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 680751 13.39 MB/sec execute 156 sec latency 26.386 ms 2024-03-14T21:19:57.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 682612 13.38 MB/sec execute 157 sec latency 63.938 ms 2024-03-14T21:19:58.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 685644 13.41 MB/sec execute 158 sec latency 15.235 ms 2024-03-14T21:19:59.409 INFO:tasks.workunit.client.0.smithi163.stdout: 1 688282 13.40 MB/sec execute 159 sec latency 6.386 ms 2024-03-14T21:20:00.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 690436 13.40 MB/sec execute 160 sec latency 20.957 ms 2024-03-14T21:20:01.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 692948 13.40 MB/sec execute 161 sec latency 7.309 ms 2024-03-14T21:20:02.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 695686 13.40 MB/sec execute 162 sec latency 61.116 ms 2024-03-14T21:20:03.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 697425 13.39 MB/sec execute 163 sec latency 66.851 ms 2024-03-14T21:20:04.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 700137 13.39 MB/sec execute 164 sec latency 5.740 ms 2024-03-14T21:20:05.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 703190 13.42 MB/sec execute 165 sec latency 6.629 ms 2024-03-14T21:20:06.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 705499 13.39 MB/sec execute 166 sec latency 22.334 ms 2024-03-14T21:20:06.739 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:20:06.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:20:06.744 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:20:07.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 707463 13.38 MB/sec execute 167 sec latency 55.931 ms 2024-03-14T21:20:08.410 INFO:tasks.workunit.client.0.smithi163.stdout: 1 710029 13.38 MB/sec execute 168 sec latency 85.479 ms 2024-03-14T21:20:09.411 INFO:tasks.workunit.client.0.smithi163.stdout: 1 712315 13.37 MB/sec execute 169 sec latency 6.158 ms 2024-03-14T21:20:10.411 INFO:tasks.workunit.client.0.smithi163.stdout: 1 714638 13.37 MB/sec execute 170 sec latency 6.228 ms 2024-03-14T21:20:11.411 INFO:tasks.workunit.client.0.smithi163.stdout: 1 717566 13.40 MB/sec execute 171 sec latency 7.718 ms 2024-03-14T21:20:12.411 INFO:tasks.workunit.client.0.smithi163.stdout: 1 719544 13.36 MB/sec execute 172 sec latency 69.094 ms 2024-03-14T21:20:13.411 INFO:tasks.workunit.client.0.smithi163.stdout: 1 721327 13.36 MB/sec execute 173 sec latency 53.640 ms 2024-03-14T21:20:14.411 INFO:tasks.workunit.client.0.smithi163.stdout: 1 724213 13.36 MB/sec execute 174 sec latency 7.098 ms 2024-03-14T21:20:15.411 INFO:tasks.workunit.client.0.smithi163.stdout: 1 726545 13.35 MB/sec execute 175 sec latency 7.477 ms 2024-03-14T21:20:16.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 728686 13.35 MB/sec execute 176 sec latency 16.899 ms 2024-03-14T21:20:17.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 731583 13.37 MB/sec execute 177 sec latency 54.847 ms 2024-03-14T21:20:18.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 733272 13.33 MB/sec execute 178 sec latency 38.486 ms 2024-03-14T21:20:19.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 735827 13.34 MB/sec execute 179 sec latency 6.219 ms 2024-03-14T21:20:20.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 738838 13.37 MB/sec execute 180 sec latency 6.248 ms 2024-03-14T21:20:21.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 741364 13.35 MB/sec execute 181 sec latency 6.609 ms 2024-03-14T21:20:22.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 742955 13.33 MB/sec execute 182 sec latency 119.371 ms 2024-03-14T21:20:23.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 745487 13.33 MB/sec execute 183 sec latency 66.263 ms 2024-03-14T21:20:24.412 INFO:tasks.workunit.client.0.smithi163.stdout: 1 747767 13.32 MB/sec execute 184 sec latency 6.252 ms 2024-03-14T21:20:25.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 750115 13.32 MB/sec execute 185 sec latency 6.574 ms 2024-03-14T21:20:26.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 753051 13.35 MB/sec execute 186 sec latency 7.278 ms 2024-03-14T21:20:27.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 755194 13.31 MB/sec execute 187 sec latency 64.591 ms 2024-03-14T21:20:28.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 756935 13.31 MB/sec execute 188 sec latency 56.017 ms 2024-03-14T21:20:29.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 759937 13.33 MB/sec execute 189 sec latency 26.145 ms 2024-03-14T21:20:30.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 762321 13.31 MB/sec execute 190 sec latency 6.166 ms 2024-03-14T21:20:31.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 764630 13.31 MB/sec execute 191 sec latency 6.136 ms 2024-03-14T21:20:32.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 767341 13.33 MB/sec execute 192 sec latency 48.532 ms 2024-03-14T21:20:33.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 769559 13.30 MB/sec execute 193 sec latency 85.413 ms 2024-03-14T21:20:34.413 INFO:tasks.workunit.client.0.smithi163.stdout: 1 771750 13.30 MB/sec execute 194 sec latency 15.786 ms 2024-03-14T21:20:35.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 774502 13.32 MB/sec execute 195 sec latency 12.819 ms 2024-03-14T21:20:36.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 777544 13.34 MB/sec execute 196 sec latency 6.987 ms 2024-03-14T21:20:37.092 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:20:37.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:20:37.097 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:20:37.132 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T21:20:37.151+0000 7f137ed27640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46137) UID: 0 2024-03-14T21:20:37.133 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-03-14T21:20:37.151+0000 7f5aad84b640 -1 received signal: Hangup from (PID: 46137) UID: 0 2024-03-14T21:20:37.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 779999 13.32 MB/sec execute 197 sec latency 7.355 ms 2024-03-14T21:20:38.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 781797 13.31 MB/sec execute 198 sec latency 101.842 ms 2024-03-14T21:20:39.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 784486 13.32 MB/sec execute 199 sec latency 29.477 ms 2024-03-14T21:20:40.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 786789 13.30 MB/sec execute 200 sec latency 7.706 ms 2024-03-14T21:20:41.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 788984 13.30 MB/sec execute 201 sec latency 20.708 ms 2024-03-14T21:20:42.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 791933 13.33 MB/sec execute 202 sec latency 7.508 ms 2024-03-14T21:20:43.414 INFO:tasks.workunit.client.0.smithi163.stdout: 1 794189 13.30 MB/sec execute 203 sec latency 43.618 ms 2024-03-14T21:20:44.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 796147 13.30 MB/sec execute 204 sec latency 46.046 ms 2024-03-14T21:20:45.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 799150 13.32 MB/sec execute 205 sec latency 6.397 ms 2024-03-14T21:20:46.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 801467 13.30 MB/sec execute 206 sec latency 22.757 ms 2024-03-14T21:20:47.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 803727 13.30 MB/sec execute 207 sec latency 6.492 ms 2024-03-14T21:20:48.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 806425 13.31 MB/sec execute 208 sec latency 7.011 ms 2024-03-14T21:20:49.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 808832 13.30 MB/sec execute 209 sec latency 40.052 ms 2024-03-14T21:20:50.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 810998 13.30 MB/sec execute 210 sec latency 6.650 ms 2024-03-14T21:20:51.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 813608 13.31 MB/sec execute 211 sec latency 22.039 ms 2024-03-14T21:20:52.416 INFO:tasks.workunit.client.0.smithi163.stdout: 1 816437 13.31 MB/sec execute 212 sec latency 6.773 ms 2024-03-14T21:20:53.415 INFO:tasks.workunit.client.0.smithi163.stdout: 1 818573 13.30 MB/sec execute 213 sec latency 6.663 ms 2024-03-14T21:20:54.416 INFO:tasks.workunit.client.0.smithi163.stdout: 1 820531 13.30 MB/sec execute 214 sec latency 50.018 ms 2024-03-14T21:20:55.416 INFO:tasks.workunit.client.0.smithi163.stdout: 1 823115 13.30 MB/sec execute 215 sec latency 7.121 ms 2024-03-14T21:20:56.416 INFO:tasks.workunit.client.0.smithi163.stdout: 1 825137 13.29 MB/sec execute 216 sec latency 16.923 ms 2024-03-14T21:20:57.416 INFO:tasks.workunit.client.0.smithi163.stdout: 1 827691 13.29 MB/sec execute 217 sec latency 7.912 ms 2024-03-14T21:20:58.416 INFO:tasks.workunit.client.0.smithi163.stdout: 1 830552 13.30 MB/sec execute 218 sec latency 8.276 ms 2024-03-14T21:20:59.416 INFO:tasks.workunit.client.0.smithi163.stdout: 1 831945 13.27 MB/sec execute 219 sec latency 47.965 ms 2024-03-14T21:21:00.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 834772 13.28 MB/sec execute 220 sec latency 6.407 ms 2024-03-14T21:21:01.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 837715 13.29 MB/sec execute 221 sec latency 6.091 ms 2024-03-14T21:21:02.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 839999 13.28 MB/sec execute 222 sec latency 31.816 ms 2024-03-14T21:21:03.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 842249 13.28 MB/sec execute 223 sec latency 6.626 ms 2024-03-14T21:21:04.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 844641 13.28 MB/sec execute 224 sec latency 87.545 ms 2024-03-14T21:21:05.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 846796 13.27 MB/sec execute 225 sec latency 6.194 ms 2024-03-14T21:21:06.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 849285 13.27 MB/sec execute 226 sec latency 6.089 ms 2024-03-14T21:21:07.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 852225 13.29 MB/sec execute 227 sec latency 6.717 ms 2024-03-14T21:21:08.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 854480 13.27 MB/sec execute 228 sec latency 34.370 ms 2024-03-14T21:21:09.417 INFO:tasks.workunit.client.0.smithi163.stdout: 1 856537 13.27 MB/sec execute 229 sec latency 6.775 ms 2024-03-14T21:21:10.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 859023 13.27 MB/sec execute 230 sec latency 103.843 ms 2024-03-14T21:21:11.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 861620 13.26 MB/sec execute 231 sec latency 6.288 ms 2024-03-14T21:21:12.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 863803 13.26 MB/sec execute 232 sec latency 40.413 ms 2024-03-14T21:21:13.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 866665 13.28 MB/sec execute 233 sec latency 33.307 ms 2024-03-14T21:21:14.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 869575 13.29 MB/sec execute 234 sec latency 7.469 ms 2024-03-14T21:21:15.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 871220 13.27 MB/sec execute 235 sec latency 51.855 ms 2024-03-14T21:21:16.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 873808 13.27 MB/sec execute 236 sec latency 6.189 ms 2024-03-14T21:21:17.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 876594 13.27 MB/sec execute 237 sec latency 20.562 ms 2024-03-14T21:21:18.418 INFO:tasks.workunit.client.0.smithi163.stdout: 1 878619 13.27 MB/sec execute 238 sec latency 17.682 ms 2024-03-14T21:21:19.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 881095 13.27 MB/sec execute 239 sec latency 6.225 ms 2024-03-14T21:21:20.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 883644 13.27 MB/sec execute 240 sec latency 33.494 ms 2024-03-14T21:21:21.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 885603 13.26 MB/sec execute 241 sec latency 6.366 ms 2024-03-14T21:21:22.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 888029 13.26 MB/sec execute 242 sec latency 6.572 ms 2024-03-14T21:21:23.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 890914 13.27 MB/sec execute 243 sec latency 6.754 ms 2024-03-14T21:21:24.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 892729 13.25 MB/sec execute 244 sec latency 34.546 ms 2024-03-14T21:21:25.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 895041 13.26 MB/sec execute 245 sec latency 54.580 ms 2024-03-14T21:21:26.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 897637 13.25 MB/sec execute 246 sec latency 7.159 ms 2024-03-14T21:21:27.419 INFO:tasks.workunit.client.0.smithi163.stdout: 1 899450 13.23 MB/sec execute 247 sec latency 10.115 ms 2024-03-14T21:21:28.420 INFO:tasks.workunit.client.0.smithi163.stdout: 1 902023 13.25 MB/sec execute 248 sec latency 19.607 ms 2024-03-14T21:21:29.420 INFO:tasks.workunit.client.0.smithi163.stdout: 1 904324 13.23 MB/sec execute 249 sec latency 26.513 ms 2024-03-14T21:21:30.420 INFO:tasks.workunit.client.0.smithi163.stdout: 1 906014 13.22 MB/sec execute 250 sec latency 46.161 ms 2024-03-14T21:21:31.420 INFO:tasks.workunit.client.0.smithi163.stdout: 1 908975 13.24 MB/sec execute 251 sec latency 10.760 ms 2024-03-14T21:21:32.420 INFO:tasks.workunit.client.0.smithi163.stdout: 1 911317 13.22 MB/sec execute 252 sec latency 27.741 ms 2024-03-14T21:21:33.420 INFO:tasks.workunit.client.0.smithi163.stdout: 1 913296 13.22 MB/sec execute 253 sec latency 43.239 ms 2024-03-14T21:21:34.420 INFO:tasks.workunit.client.0.smithi163.stdout: 1 916194 13.24 MB/sec execute 254 sec latency 7.575 ms 2024-03-14T21:21:35.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 917268 13.26 MB/sec execute 255 sec latency 66.200 ms 2024-03-14T21:21:36.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 920321 13.29 MB/sec execute 256 sec latency 6.507 ms 2024-03-14T21:21:37.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 922649 13.27 MB/sec execute 257 sec latency 6.447 ms 2024-03-14T21:21:38.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 924599 13.26 MB/sec execute 258 sec latency 60.018 ms 2024-03-14T21:21:39.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 927493 13.28 MB/sec execute 259 sec latency 20.859 ms 2024-03-14T21:21:40.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 929170 13.25 MB/sec execute 260 sec latency 111.384 ms 2024-03-14T21:21:41.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 931565 13.26 MB/sec execute 261 sec latency 10.505 ms 2024-03-14T21:21:42.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 934545 13.27 MB/sec execute 262 sec latency 14.607 ms 2024-03-14T21:21:43.421 INFO:tasks.workunit.client.0.smithi163.stdout: 1 936920 13.26 MB/sec execute 263 sec latency 6.800 ms 2024-03-14T21:21:44.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 938991 13.25 MB/sec execute 264 sec latency 23.858 ms 2024-03-14T21:21:45.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 941771 13.27 MB/sec execute 265 sec latency 77.798 ms 2024-03-14T21:21:46.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 944005 13.25 MB/sec execute 266 sec latency 19.930 ms 2024-03-14T21:21:47.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 946257 13.25 MB/sec execute 267 sec latency 7.560 ms 2024-03-14T21:21:48.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 949065 13.27 MB/sec execute 268 sec latency 5.842 ms 2024-03-14T21:21:49.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 951662 13.26 MB/sec execute 269 sec latency 7.614 ms 2024-03-14T21:21:50.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 953715 13.26 MB/sec execute 270 sec latency 19.509 ms 2024-03-14T21:21:51.422 INFO:tasks.workunit.client.0.smithi163.stdout: 1 956066 13.26 MB/sec execute 271 sec latency 74.329 ms 2024-03-14T21:21:52.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 958511 13.25 MB/sec execute 272 sec latency 80.420 ms 2024-03-14T21:21:53.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 960265 13.24 MB/sec execute 273 sec latency 7.140 ms 2024-03-14T21:21:54.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 963224 13.26 MB/sec execute 274 sec latency 6.016 ms 2024-03-14T21:21:55.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 966006 13.26 MB/sec execute 275 sec latency 7.325 ms 2024-03-14T21:21:56.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 967788 13.25 MB/sec execute 276 sec latency 43.645 ms 2024-03-14T21:21:57.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 970349 13.25 MB/sec execute 277 sec latency 15.573 ms 2024-03-14T21:21:58.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 972947 13.25 MB/sec execute 278 sec latency 6.727 ms 2024-03-14T21:21:59.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 975383 13.24 MB/sec execute 279 sec latency 6.737 ms 2024-03-14T21:22:00.423 INFO:tasks.workunit.client.0.smithi163.stdout: 1 977807 13.25 MB/sec execute 280 sec latency 24.265 ms 2024-03-14T21:22:01.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 980409 13.25 MB/sec execute 281 sec latency 90.762 ms 2024-03-14T21:22:02.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 982473 13.24 MB/sec execute 282 sec latency 13.781 ms 2024-03-14T21:22:03.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 984846 13.24 MB/sec execute 283 sec latency 6.539 ms 2024-03-14T21:22:04.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 987894 13.26 MB/sec execute 284 sec latency 6.601 ms 2024-03-14T21:22:05.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 990402 13.25 MB/sec execute 285 sec latency 6.626 ms 2024-03-14T21:22:06.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 991954 13.24 MB/sec execute 286 sec latency 54.992 ms 2024-03-14T21:22:07.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 994860 13.25 MB/sec execute 287 sec latency 26.411 ms 2024-03-14T21:22:08.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 997143 13.23 MB/sec execute 288 sec latency 23.069 ms 2024-03-14T21:22:09.424 INFO:tasks.workunit.client.0.smithi163.stdout: 1 999237 13.23 MB/sec execute 289 sec latency 6.926 ms 2024-03-14T21:22:10.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1002125 13.25 MB/sec execute 290 sec latency 6.726 ms 2024-03-14T21:22:11.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1004343 13.23 MB/sec execute 291 sec latency 15.898 ms 2024-03-14T21:22:12.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1006136 13.23 MB/sec execute 292 sec latency 103.055 ms 2024-03-14T21:22:13.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1008899 13.24 MB/sec execute 293 sec latency 29.909 ms 2024-03-14T21:22:14.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1011049 13.22 MB/sec execute 294 sec latency 13.741 ms 2024-03-14T21:22:15.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1013171 13.22 MB/sec execute 295 sec latency 17.347 ms 2024-03-14T21:22:16.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1015867 13.22 MB/sec execute 296 sec latency 7.117 ms 2024-03-14T21:22:17.425 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1017565 13.21 MB/sec execute 297 sec latency 55.267 ms 2024-03-14T21:22:18.426 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1020007 13.21 MB/sec execute 298 sec latency 34.089 ms 2024-03-14T21:22:19.426 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1022568 13.21 MB/sec execute 299 sec latency 11.444 ms 2024-03-14T21:22:20.426 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1024541 13.21 MB/sec execute 300 sec latency 24.746 ms 2024-03-14T21:22:21.426 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1027104 13.21 MB/sec execute 301 sec latency 7.178 ms 2024-03-14T21:22:22.426 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1029278 13.19 MB/sec execute 302 sec latency 91.889 ms 2024-03-14T21:22:23.426 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1031168 13.19 MB/sec execute 303 sec latency 39.959 ms 2024-03-14T21:22:24.427 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1034007 13.20 MB/sec execute 304 sec latency 9.191 ms 2024-03-14T21:22:25.427 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1036180 13.19 MB/sec execute 305 sec latency 24.810 ms 2024-03-14T21:22:26.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1038122 13.18 MB/sec execute 306 sec latency 6.824 ms 2024-03-14T21:22:27.427 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1040708 13.19 MB/sec execute 307 sec latency 45.808 ms 2024-03-14T21:22:28.427 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1042650 13.18 MB/sec execute 308 sec latency 23.730 ms 2024-03-14T21:22:29.427 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1045205 13.18 MB/sec execute 309 sec latency 6.087 ms 2024-03-14T21:22:30.427 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1048069 13.19 MB/sec execute 310 sec latency 15.586 ms 2024-03-14T21:22:31.427 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1050371 13.18 MB/sec execute 311 sec latency 11.775 ms 2024-03-14T21:22:32.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1052056 13.17 MB/sec execute 312 sec latency 102.477 ms 2024-03-14T21:22:33.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1054517 13.17 MB/sec execute 313 sec latency 37.176 ms 2024-03-14T21:22:34.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1056588 13.17 MB/sec execute 314 sec latency 7.078 ms 2024-03-14T21:22:35.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1059183 13.17 MB/sec execute 315 sec latency 11.107 ms 2024-03-14T21:22:36.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1061967 13.17 MB/sec execute 316 sec latency 11.639 ms 2024-03-14T21:22:37.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1063706 13.16 MB/sec execute 317 sec latency 84.651 ms 2024-03-14T21:22:38.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1066201 13.17 MB/sec execute 318 sec latency 31.512 ms 2024-03-14T21:22:39.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1068987 13.17 MB/sec execute 319 sec latency 8.022 ms 2024-03-14T21:22:40.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1071130 13.16 MB/sec execute 320 sec latency 7.730 ms 2024-03-14T21:22:41.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1073462 13.16 MB/sec execute 321 sec latency 7.571 ms 2024-03-14T21:22:42.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1075779 13.16 MB/sec execute 322 sec latency 75.778 ms 2024-03-14T21:22:43.428 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1077799 13.15 MB/sec execute 323 sec latency 8.707 ms 2024-03-14T21:22:44.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1080305 13.16 MB/sec execute 324 sec latency 6.635 ms 2024-03-14T21:22:45.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1082849 13.16 MB/sec execute 325 sec latency 6.999 ms 2024-03-14T21:22:46.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1084822 13.15 MB/sec execute 326 sec latency 10.752 ms 2024-03-14T21:22:47.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1087216 13.15 MB/sec execute 327 sec latency 68.875 ms 2024-03-14T21:22:48.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1089452 13.14 MB/sec execute 328 sec latency 26.824 ms 2024-03-14T21:22:49.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1091490 13.13 MB/sec execute 329 sec latency 6.951 ms 2024-03-14T21:22:50.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1094339 13.15 MB/sec execute 330 sec latency 7.081 ms 2024-03-14T21:22:51.429 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1096930 13.14 MB/sec execute 331 sec latency 11.965 ms 2024-03-14T21:22:52.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1098580 13.13 MB/sec execute 332 sec latency 51.806 ms 2024-03-14T21:22:53.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1101284 13.14 MB/sec execute 333 sec latency 54.674 ms 2024-03-14T21:22:54.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1103551 13.13 MB/sec execute 334 sec latency 7.246 ms 2024-03-14T21:22:55.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1105696 13.13 MB/sec execute 335 sec latency 7.640 ms 2024-03-14T21:22:56.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1108607 13.14 MB/sec execute 336 sec latency 6.984 ms 2024-03-14T21:22:57.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1111252 13.14 MB/sec execute 337 sec latency 28.729 ms 2024-03-14T21:22:58.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1113187 13.14 MB/sec execute 338 sec latency 120.584 ms 2024-03-14T21:22:59.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1115791 13.14 MB/sec execute 339 sec latency 33.805 ms 2024-03-14T21:23:00.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1118861 13.15 MB/sec execute 340 sec latency 6.704 ms 2024-03-14T21:23:01.430 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1121311 13.14 MB/sec execute 341 sec latency 11.071 ms 2024-03-14T21:23:02.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1123490 13.14 MB/sec execute 342 sec latency 11.433 ms 2024-03-14T21:23:03.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1126266 13.15 MB/sec execute 343 sec latency 55.250 ms 2024-03-14T21:23:04.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1128732 13.15 MB/sec execute 344 sec latency 32.438 ms 2024-03-14T21:23:05.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1130960 13.15 MB/sec execute 345 sec latency 5.913 ms 2024-03-14T21:23:06.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1133644 13.15 MB/sec execute 346 sec latency 7.358 ms 2024-03-14T21:23:07.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1136450 13.15 MB/sec execute 347 sec latency 21.774 ms 2024-03-14T21:23:08.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1138211 13.14 MB/sec execute 348 sec latency 40.651 ms 2024-03-14T21:23:09.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1140680 13.15 MB/sec execute 349 sec latency 26.125 ms 2024-03-14T21:23:10.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1143571 13.15 MB/sec execute 350 sec latency 13.659 ms 2024-03-14T21:23:11.431 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1145861 13.14 MB/sec execute 351 sec latency 6.605 ms 2024-03-14T21:23:12.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1148025 13.14 MB/sec execute 352 sec latency 9.788 ms 2024-03-14T21:23:13.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1150500 13.14 MB/sec execute 353 sec latency 51.188 ms 2024-03-14T21:23:14.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1152444 13.14 MB/sec execute 354 sec latency 32.272 ms 2024-03-14T21:23:15.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1154823 13.14 MB/sec execute 355 sec latency 47.758 ms 2024-03-14T21:23:16.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1157405 13.14 MB/sec execute 356 sec latency 19.204 ms 2024-03-14T21:23:17.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1159565 13.13 MB/sec execute 357 sec latency 7.836 ms 2024-03-14T21:23:18.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1161767 13.14 MB/sec execute 358 sec latency 94.883 ms 2024-03-14T21:23:19.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1163904 13.12 MB/sec execute 359 sec latency 43.648 ms 2024-03-14T21:23:20.432 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1165891 13.12 MB/sec execute 360 sec latency 8.418 ms 2024-03-14T21:23:21.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1168641 13.13 MB/sec execute 361 sec latency 13.615 ms 2024-03-14T21:23:22.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1170914 13.12 MB/sec execute 362 sec latency 12.956 ms 2024-03-14T21:23:23.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1172640 13.11 MB/sec execute 363 sec latency 73.463 ms 2024-03-14T21:23:24.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1175276 13.11 MB/sec execute 364 sec latency 34.321 ms 2024-03-14T21:23:25.233 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:23:25.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:23:25.239 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:23:25.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1177399 13.11 MB/sec execute 365 sec latency 9.647 ms 2024-03-14T21:23:26.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1179794 13.11 MB/sec execute 366 sec latency 11.495 ms 2024-03-14T21:23:27.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1182712 13.12 MB/sec execute 367 sec latency 6.837 ms 2024-03-14T21:23:28.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1184512 13.11 MB/sec execute 368 sec latency 41.976 ms 2024-03-14T21:23:29.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1186828 13.11 MB/sec execute 369 sec latency 30.988 ms 2024-03-14T21:23:30.433 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1189666 13.11 MB/sec execute 370 sec latency 7.155 ms 2024-03-14T21:23:31.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1192040 13.10 MB/sec execute 371 sec latency 6.355 ms 2024-03-14T21:23:32.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1194149 13.11 MB/sec execute 372 sec latency 21.381 ms 2024-03-14T21:23:33.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1196891 13.12 MB/sec execute 373 sec latency 60.780 ms 2024-03-14T21:23:34.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1198436 13.10 MB/sec execute 374 sec latency 45.268 ms 2024-03-14T21:23:35.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1201055 13.10 MB/sec execute 375 sec latency 5.936 ms 2024-03-14T21:23:36.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1204096 13.11 MB/sec execute 376 sec latency 6.345 ms 2024-03-14T21:23:37.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1206416 13.10 MB/sec execute 377 sec latency 19.797 ms 2024-03-14T21:23:38.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1208447 13.10 MB/sec execute 378 sec latency 9.115 ms 2024-03-14T21:23:39.434 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1211215 13.11 MB/sec execute 379 sec latency 67.285 ms 2024-03-14T21:23:40.435 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1213599 13.10 MB/sec execute 380 sec latency 5.677 ms 2024-03-14T21:23:41.435 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1215917 13.10 MB/sec execute 381 sec latency 6.626 ms 2024-03-14T21:23:42.435 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1218657 13.11 MB/sec execute 382 sec latency 9.348 ms 2024-03-14T21:23:43.435 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1221412 13.11 MB/sec execute 383 sec latency 27.323 ms 2024-03-14T21:23:44.435 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1223156 13.11 MB/sec execute 384 sec latency 72.463 ms 2024-03-14T21:23:45.435 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1225724 13.11 MB/sec execute 385 sec latency 18.775 ms 2024-03-14T21:23:46.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1228511 13.11 MB/sec execute 386 sec latency 6.474 ms 2024-03-14T21:23:47.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1230907 13.11 MB/sec execute 387 sec latency 7.330 ms 2024-03-14T21:23:48.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1233182 13.11 MB/sec execute 388 sec latency 25.729 ms 2024-03-14T21:23:49.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1235933 13.12 MB/sec execute 389 sec latency 56.248 ms 2024-03-14T21:23:50.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1237826 13.10 MB/sec execute 390 sec latency 32.419 ms 2024-03-14T21:23:51.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1240097 13.10 MB/sec execute 391 sec latency 6.892 ms 2024-03-14T21:23:52.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1243038 13.11 MB/sec execute 392 sec latency 14.215 ms 2024-03-14T21:23:53.436 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1245230 13.10 MB/sec execute 393 sec latency 27.272 ms 2024-03-14T21:23:54.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1247226 13.10 MB/sec execute 394 sec latency 46.763 ms 2024-03-14T21:23:55.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1250206 13.11 MB/sec execute 395 sec latency 26.742 ms 2024-03-14T21:23:55.504 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:23:55.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:23:55.510 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:23:56.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1252513 13.10 MB/sec execute 396 sec latency 54.262 ms 2024-03-14T21:23:57.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1254608 13.10 MB/sec execute 397 sec latency 6.950 ms 2024-03-14T21:23:58.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1257543 13.11 MB/sec execute 398 sec latency 21.343 ms 2024-03-14T21:23:59.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1259685 13.10 MB/sec execute 399 sec latency 66.465 ms 2024-03-14T21:24:00.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1261740 13.10 MB/sec execute 400 sec latency 20.137 ms 2024-03-14T21:24:01.437 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1264683 13.11 MB/sec execute 401 sec latency 6.389 ms 2024-03-14T21:24:02.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1267471 13.11 MB/sec execute 402 sec latency 6.772 ms 2024-03-14T21:24:03.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1269622 13.11 MB/sec execute 403 sec latency 28.602 ms 2024-03-14T21:24:04.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1271866 13.11 MB/sec execute 404 sec latency 70.080 ms 2024-03-14T21:24:05.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1274563 13.11 MB/sec execute 405 sec latency 21.481 ms 2024-03-14T21:24:06.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1276723 13.10 MB/sec execute 406 sec latency 6.779 ms 2024-03-14T21:24:07.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1279141 13.11 MB/sec execute 407 sec latency 11.549 ms 2024-03-14T21:24:08.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1281808 13.11 MB/sec execute 408 sec latency 21.033 ms 2024-03-14T21:24:09.438 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1283215 13.09 MB/sec execute 409 sec latency 125.918 ms 2024-03-14T21:24:10.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1285899 13.10 MB/sec execute 410 sec latency 31.251 ms 2024-03-14T21:24:11.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1288593 13.10 MB/sec execute 411 sec latency 6.460 ms 2024-03-14T21:24:12.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1290823 13.10 MB/sec execute 412 sec latency 6.110 ms 2024-03-14T21:24:13.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1293373 13.10 MB/sec execute 413 sec latency 7.665 ms 2024-03-14T21:24:14.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1296040 13.10 MB/sec execute 414 sec latency 66.975 ms 2024-03-14T21:24:15.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1298348 13.10 MB/sec execute 415 sec latency 23.529 ms 2024-03-14T21:24:16.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1300582 13.10 MB/sec execute 416 sec latency 9.550 ms 2024-03-14T21:24:17.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1303596 13.11 MB/sec execute 417 sec latency 6.552 ms 2024-03-14T21:24:18.439 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1305925 13.10 MB/sec execute 418 sec latency 7.409 ms 2024-03-14T21:24:19.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1307791 13.10 MB/sec execute 419 sec latency 59.065 ms 2024-03-14T21:24:20.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1310270 13.10 MB/sec execute 420 sec latency 81.215 ms 2024-03-14T21:24:21.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1312678 13.09 MB/sec execute 421 sec latency 6.245 ms 2024-03-14T21:24:22.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1314911 13.09 MB/sec execute 422 sec latency 7.285 ms 2024-03-14T21:24:23.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1317854 13.11 MB/sec execute 423 sec latency 6.921 ms 2024-03-14T21:24:24.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1320323 13.10 MB/sec execute 424 sec latency 7.811 ms 2024-03-14T21:24:25.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1322002 13.09 MB/sec execute 425 sec latency 43.234 ms 2024-03-14T21:24:25.851 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:24:25.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:24:25.856 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:24:26.440 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1324925 13.10 MB/sec execute 426 sec latency 8.621 ms 2024-03-14T21:24:27.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1327405 13.10 MB/sec execute 427 sec latency 7.431 ms 2024-03-14T21:24:28.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1329400 13.09 MB/sec execute 428 sec latency 6.631 ms 2024-03-14T21:24:29.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1332057 13.10 MB/sec execute 429 sec latency 19.913 ms 2024-03-14T21:24:30.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1334064 13.09 MB/sec execute 430 sec latency 43.491 ms 2024-03-14T21:24:31.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1335809 13.09 MB/sec execute 431 sec latency 91.393 ms 2024-03-14T21:24:32.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1338565 13.09 MB/sec execute 432 sec latency 13.025 ms 2024-03-14T21:24:33.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1340515 13.08 MB/sec execute 433 sec latency 7.119 ms 2024-03-14T21:24:34.441 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1342867 13.08 MB/sec execute 434 sec latency 7.624 ms 2024-03-14T21:24:35.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1344964 13.07 MB/sec execute 435 sec latency 77.519 ms 2024-03-14T21:24:36.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1346784 13.07 MB/sec execute 436 sec latency 56.974 ms 2024-03-14T21:24:37.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1349599 13.08 MB/sec execute 437 sec latency 8.661 ms 2024-03-14T21:24:38.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1351848 13.07 MB/sec execute 438 sec latency 7.765 ms 2024-03-14T21:24:39.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1353781 13.07 MB/sec execute 439 sec latency 7.160 ms 2024-03-14T21:24:40.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1356370 13.07 MB/sec execute 440 sec latency 63.870 ms 2024-03-14T21:24:41.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1358062 13.06 MB/sec execute 441 sec latency 38.432 ms 2024-03-14T21:24:42.442 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1360534 13.06 MB/sec execute 442 sec latency 10.963 ms 2024-03-14T21:24:43.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1363261 13.06 MB/sec execute 443 sec latency 16.687 ms 2024-03-14T21:24:44.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1365293 13.06 MB/sec execute 444 sec latency 27.463 ms 2024-03-14T21:24:45.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1367582 13.06 MB/sec execute 445 sec latency 37.408 ms 2024-03-14T21:24:46.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1370176 13.06 MB/sec execute 446 sec latency 36.296 ms 2024-03-14T21:24:47.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1372385 13.06 MB/sec execute 447 sec latency 6.364 ms 2024-03-14T21:24:48.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1374941 13.06 MB/sec execute 448 sec latency 7.870 ms 2024-03-14T21:24:49.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1376646 13.09 MB/sec execute 449 sec latency 55.334 ms 2024-03-14T21:24:50.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1378929 13.09 MB/sec execute 450 sec latency 79.612 ms 2024-03-14T21:24:51.443 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1381088 13.08 MB/sec execute 451 sec latency 33.625 ms 2024-03-14T21:24:52.444 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1383057 13.08 MB/sec execute 452 sec latency 14.727 ms 2024-03-14T21:24:53.444 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1385818 13.08 MB/sec execute 453 sec latency 7.116 ms 2024-03-14T21:24:54.444 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1387933 13.07 MB/sec execute 454 sec latency 19.412 ms 2024-03-14T21:24:55.444 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1389703 13.07 MB/sec execute 455 sec latency 90.679 ms 2024-03-14T21:24:55.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:24:55.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:24:55.978 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:24:56.444 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1392436 13.07 MB/sec execute 456 sec latency 34.330 ms 2024-03-14T21:24:57.444 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1394705 13.07 MB/sec execute 457 sec latency 7.857 ms 2024-03-14T21:24:58.444 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1397137 13.07 MB/sec execute 458 sec latency 6.690 ms 2024-03-14T21:24:59.445 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1400025 13.08 MB/sec execute 459 sec latency 36.046 ms 2024-03-14T21:25:00.445 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1402167 13.07 MB/sec execute 460 sec latency 37.367 ms 2024-03-14T21:25:01.445 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1404170 13.07 MB/sec execute 461 sec latency 30.114 ms 2024-03-14T21:25:02.445 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1407083 13.08 MB/sec execute 462 sec latency 8.616 ms 2024-03-14T21:25:03.445 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1409437 13.07 MB/sec execute 463 sec latency 7.377 ms 2024-03-14T21:25:04.445 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1411509 13.07 MB/sec execute 464 sec latency 29.391 ms 2024-03-14T21:25:05.445 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1414387 13.08 MB/sec execute 465 sec latency 6.786 ms 2024-03-14T21:25:06.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1416290 13.06 MB/sec execute 466 sec latency 42.940 ms 2024-03-14T21:25:07.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1418212 13.06 MB/sec execute 467 sec latency 7.739 ms 2024-03-14T21:25:08.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1420602 13.06 MB/sec execute 468 sec latency 8.974 ms 2024-03-14T21:25:09.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1422430 13.05 MB/sec execute 469 sec latency 7.703 ms 2024-03-14T21:25:10.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1424999 13.06 MB/sec execute 470 sec latency 19.830 ms 2024-03-14T21:25:11.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1426548 13.05 MB/sec execute 471 sec latency 88.401 ms 2024-03-14T21:25:12.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1428936 13.05 MB/sec execute 472 sec latency 7.195 ms 2024-03-14T21:25:13.446 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1431886 13.06 MB/sec execute 473 sec latency 6.977 ms 2024-03-14T21:25:14.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1433894 13.04 MB/sec execute 474 sec latency 8.776 ms 2024-03-14T21:25:15.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1435919 13.04 MB/sec execute 475 sec latency 27.148 ms 2024-03-14T21:25:16.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1438118 13.04 MB/sec execute 476 sec latency 42.544 ms 2024-03-14T21:25:17.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1440152 13.03 MB/sec execute 477 sec latency 7.351 ms 2024-03-14T21:25:18.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1442868 13.04 MB/sec execute 478 sec latency 6.398 ms 2024-03-14T21:25:19.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1445618 13.04 MB/sec execute 479 sec latency 6.951 ms 2024-03-14T21:25:20.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1447554 13.04 MB/sec execute 480 sec latency 21.031 ms 2024-03-14T21:25:21.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1449856 13.04 MB/sec execute 481 sec latency 91.132 ms 2024-03-14T21:25:22.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1452157 13.03 MB/sec execute 482 sec latency 30.308 ms 2024-03-14T21:25:23.447 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1454273 13.03 MB/sec execute 483 sec latency 7.394 ms 2024-03-14T21:25:24.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1457021 13.04 MB/sec execute 484 sec latency 7.034 ms 2024-03-14T21:25:25.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1459426 13.04 MB/sec execute 485 sec latency 22.307 ms 2024-03-14T21:25:26.118 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:25:26.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:25:26.123 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:25:26.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1461280 13.03 MB/sec execute 486 sec latency 7.391 ms 2024-03-14T21:25:27.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1463737 13.03 MB/sec execute 487 sec latency 63.770 ms 2024-03-14T21:25:28.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1466052 13.02 MB/sec execute 488 sec latency 6.944 ms 2024-03-14T21:25:29.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1468275 13.02 MB/sec execute 489 sec latency 5.971 ms 2024-03-14T21:25:30.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1471173 13.04 MB/sec execute 490 sec latency 6.937 ms 2024-03-14T21:25:31.448 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1473745 13.03 MB/sec execute 491 sec latency 23.767 ms 2024-03-14T21:25:32.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1475501 13.03 MB/sec execute 492 sec latency 43.074 ms 2024-03-14T21:25:33.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1478223 13.03 MB/sec execute 493 sec latency 29.388 ms 2024-03-14T21:25:34.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1480921 13.03 MB/sec execute 494 sec latency 6.677 ms 2024-03-14T21:25:35.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1483258 13.03 MB/sec execute 495 sec latency 6.463 ms 2024-03-14T21:25:36.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1485593 13.03 MB/sec execute 496 sec latency 28.656 ms 2024-03-14T21:25:37.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1488040 13.03 MB/sec execute 497 sec latency 34.991 ms 2024-03-14T21:25:38.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1490213 13.03 MB/sec execute 498 sec latency 33.988 ms 2024-03-14T21:25:39.449 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1492798 13.03 MB/sec execute 499 sec latency 6.463 ms 2024-03-14T21:25:40.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1495890 13.04 MB/sec execute 500 sec latency 6.232 ms 2024-03-14T21:25:41.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1498150 13.03 MB/sec execute 501 sec latency 21.673 ms 2024-03-14T21:25:42.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1500089 13.03 MB/sec execute 502 sec latency 70.009 ms 2024-03-14T21:25:43.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1503008 13.04 MB/sec execute 503 sec latency 25.636 ms 2024-03-14T21:25:44.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1505638 13.04 MB/sec execute 504 sec latency 7.690 ms 2024-03-14T21:25:45.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1507866 13.04 MB/sec execute 505 sec latency 6.260 ms 2024-03-14T21:25:46.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1510377 13.04 MB/sec execute 506 sec latency 15.194 ms 2024-03-14T21:25:47.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1513139 13.04 MB/sec execute 507 sec latency 35.229 ms 2024-03-14T21:25:48.450 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1514852 13.04 MB/sec execute 508 sec latency 35.790 ms 2024-03-14T21:25:49.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1517491 13.04 MB/sec execute 509 sec latency 17.662 ms 2024-03-14T21:25:50.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1520469 13.05 MB/sec execute 510 sec latency 7.215 ms 2024-03-14T21:25:51.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1522828 13.04 MB/sec execute 511 sec latency 40.851 ms 2024-03-14T21:25:52.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1524916 13.04 MB/sec execute 512 sec latency 6.955 ms 2024-03-14T21:25:53.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1527404 13.04 MB/sec execute 513 sec latency 45.570 ms 2024-03-14T21:25:54.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1529689 13.03 MB/sec execute 514 sec latency 6.583 ms 2024-03-14T21:25:55.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1531882 13.03 MB/sec execute 515 sec latency 19.565 ms 2024-03-14T21:25:56.264 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:25:56.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:25:56.268 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:25:56.451 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1534905 13.05 MB/sec execute 516 sec latency 6.384 ms 2024-03-14T21:25:57.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1537243 13.04 MB/sec execute 517 sec latency 6.627 ms 2024-03-14T21:25:58.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1538996 13.03 MB/sec execute 518 sec latency 73.660 ms 2024-03-14T21:25:59.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1541983 13.04 MB/sec execute 519 sec latency 7.320 ms 2024-03-14T21:26:00.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1544219 13.03 MB/sec execute 520 sec latency 9.666 ms 2024-03-14T21:26:01.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1546363 13.03 MB/sec execute 521 sec latency 23.823 ms 2024-03-14T21:26:02.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1549180 13.04 MB/sec execute 522 sec latency 9.831 ms 2024-03-14T21:26:03.452 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1551272 13.03 MB/sec execute 523 sec latency 45.902 ms 2024-03-14T21:26:04.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1553376 13.03 MB/sec execute 524 sec latency 6.870 ms 2024-03-14T21:26:05.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1556270 13.04 MB/sec execute 525 sec latency 6.091 ms 2024-03-14T21:26:06.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1558705 13.04 MB/sec execute 526 sec latency 33.964 ms 2024-03-14T21:26:07.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1560873 13.04 MB/sec execute 527 sec latency 12.174 ms 2024-03-14T21:26:08.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1563304 13.04 MB/sec execute 528 sec latency 54.690 ms 2024-03-14T21:26:09.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1565597 13.03 MB/sec execute 529 sec latency 6.945 ms 2024-03-14T21:26:10.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1567714 13.03 MB/sec execute 530 sec latency 7.527 ms 2024-03-14T21:26:11.453 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1570421 13.04 MB/sec execute 531 sec latency 33.064 ms 2024-03-14T21:26:12.454 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1573003 13.03 MB/sec execute 532 sec latency 7.984 ms 2024-03-14T21:26:13.454 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1574644 13.03 MB/sec execute 533 sec latency 59.324 ms 2024-03-14T21:26:14.454 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1577483 13.04 MB/sec execute 534 sec latency 26.552 ms 2024-03-14T21:26:15.454 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1579977 13.03 MB/sec execute 535 sec latency 7.015 ms 2024-03-14T21:26:16.454 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1581948 13.03 MB/sec execute 536 sec latency 7.577 ms 2024-03-14T21:26:17.454 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1584702 13.03 MB/sec execute 537 sec latency 19.685 ms 2024-03-14T21:26:18.454 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1587028 13.03 MB/sec execute 538 sec latency 57.642 ms 2024-03-14T21:26:19.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1588958 13.02 MB/sec execute 539 sec latency 34.630 ms 2024-03-14T21:26:20.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1591766 13.03 MB/sec execute 540 sec latency 6.724 ms 2024-03-14T21:26:21.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1594369 13.03 MB/sec execute 541 sec latency 6.878 ms 2024-03-14T21:26:22.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1596469 13.03 MB/sec execute 542 sec latency 14.368 ms 2024-03-14T21:26:23.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1598846 13.03 MB/sec execute 543 sec latency 67.075 ms 2024-03-14T21:26:24.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1601141 13.02 MB/sec execute 544 sec latency 25.565 ms 2024-03-14T21:26:25.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1603363 13.02 MB/sec execute 545 sec latency 6.771 ms 2024-03-14T21:26:26.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:26:26.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:26:26.405 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:26:26.437 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T21:26:26.456+0000 7f137ed27640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46283) UID: 0 2024-03-14T21:26:26.437 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-03-14T21:26:26.456+0000 7f5aad84b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46283) UID: 0 2024-03-14T21:26:26.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1606008 13.03 MB/sec execute 546 sec latency 79.446 ms 2024-03-14T21:26:27.455 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1608635 13.03 MB/sec execute 547 sec latency 10.845 ms 2024-03-14T21:26:28.456 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1610589 13.03 MB/sec execute 548 sec latency 42.018 ms 2024-03-14T21:26:29.456 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1613029 13.03 MB/sec execute 549 sec latency 70.220 ms 2024-03-14T21:26:30.456 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1615323 13.02 MB/sec execute 550 sec latency 8.826 ms 2024-03-14T21:26:31.456 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1617263 13.02 MB/sec execute 551 sec latency 19.967 ms 2024-03-14T21:26:32.456 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1620079 13.03 MB/sec execute 552 sec latency 11.542 ms 2024-03-14T21:26:33.456 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1622267 13.02 MB/sec execute 553 sec latency 24.451 ms 2024-03-14T21:26:34.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1624164 13.01 MB/sec execute 554 sec latency 99.435 ms 2024-03-14T21:26:35.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1627266 13.02 MB/sec execute 555 sec latency 7.114 ms 2024-03-14T21:26:36.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1630084 13.03 MB/sec execute 556 sec latency 9.909 ms 2024-03-14T21:26:37.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1631952 13.02 MB/sec execute 557 sec latency 21.957 ms 2024-03-14T21:26:38.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1634401 13.02 MB/sec execute 558 sec latency 21.847 ms 2024-03-14T21:26:39.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1636479 13.01 MB/sec execute 559 sec latency 64.185 ms 2024-03-14T21:26:40.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1638724 13.01 MB/sec execute 560 sec latency 7.474 ms 2024-03-14T21:26:41.458 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1641480 13.02 MB/sec execute 561 sec latency 6.872 ms 2024-03-14T21:26:42.457 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1644115 13.02 MB/sec execute 562 sec latency 23.225 ms 2024-03-14T21:26:43.458 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1646059 13.02 MB/sec execute 563 sec latency 30.459 ms 2024-03-14T21:26:44.458 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1648435 13.02 MB/sec execute 564 sec latency 53.455 ms 2024-03-14T21:26:45.458 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1650951 13.02 MB/sec execute 565 sec latency 7.141 ms 2024-03-14T21:26:46.458 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1652887 13.01 MB/sec execute 566 sec latency 6.374 ms 2024-03-14T21:26:47.459 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1655653 13.02 MB/sec execute 567 sec latency 15.009 ms 2024-03-14T21:26:48.459 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1658051 13.02 MB/sec execute 568 sec latency 8.819 ms 2024-03-14T21:26:49.459 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1659493 13.01 MB/sec execute 569 sec latency 79.754 ms 2024-03-14T21:26:50.459 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1662450 13.01 MB/sec execute 570 sec latency 11.958 ms 2024-03-14T21:26:51.459 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1664916 13.01 MB/sec execute 571 sec latency 7.676 ms 2024-03-14T21:26:52.459 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1666888 13.01 MB/sec execute 572 sec latency 6.891 ms 2024-03-14T21:26:53.459 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1669773 13.02 MB/sec execute 573 sec latency 7.456 ms 2024-03-14T21:26:54.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1671793 13.01 MB/sec execute 574 sec latency 72.538 ms 2024-03-14T21:26:55.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1673811 13.00 MB/sec execute 575 sec latency 9.265 ms 2024-03-14T21:26:56.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1676787 13.01 MB/sec execute 576 sec latency 6.646 ms 2024-03-14T21:26:57.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1679108 13.01 MB/sec execute 577 sec latency 7.414 ms 2024-03-14T21:26:58.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1681011 13.00 MB/sec execute 578 sec latency 28.969 ms 2024-03-14T21:26:59.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1683714 13.01 MB/sec execute 579 sec latency 52.390 ms 2024-03-14T21:27:00.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1685905 13.00 MB/sec execute 580 sec latency 38.688 ms 2024-03-14T21:27:01.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1688061 13.00 MB/sec execute 581 sec latency 11.884 ms 2024-03-14T21:27:02.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1691004 13.01 MB/sec execute 582 sec latency 6.218 ms 2024-03-14T21:27:03.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1693635 13.01 MB/sec execute 583 sec latency 6.294 ms 2024-03-14T21:27:04.460 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1695153 13.00 MB/sec execute 584 sec latency 127.677 ms 2024-03-14T21:27:05.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1698067 13.01 MB/sec execute 585 sec latency 25.540 ms 2024-03-14T21:27:06.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1700541 13.01 MB/sec execute 586 sec latency 7.176 ms 2024-03-14T21:27:07.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1702590 13.00 MB/sec execute 587 sec latency 9.830 ms 2024-03-14T21:27:08.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1705472 13.01 MB/sec execute 588 sec latency 6.039 ms 2024-03-14T21:27:09.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1708124 13.01 MB/sec execute 589 sec latency 80.479 ms 2024-03-14T21:27:10.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1709895 13.01 MB/sec execute 590 sec latency 42.785 ms 2024-03-14T21:27:11.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1712602 13.01 MB/sec execute 591 sec latency 6.472 ms 2024-03-14T21:27:12.461 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1715402 13.01 MB/sec execute 592 sec latency 29.186 ms 2024-03-14T21:27:13.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1717919 13.01 MB/sec execute 593 sec latency 7.138 ms 2024-03-14T21:27:14.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1720012 13.01 MB/sec execute 594 sec latency 7.017 ms 2024-03-14T21:27:15.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1722606 13.01 MB/sec execute 595 sec latency 62.781 ms 2024-03-14T21:27:16.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1725010 13.01 MB/sec execute 596 sec latency 10.255 ms 2024-03-14T21:27:17.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1727073 13.01 MB/sec execute 597 sec latency 27.162 ms 2024-03-14T21:27:18.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1730003 13.01 MB/sec execute 598 sec latency 6.847 ms 2024-03-14T21:27:19.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 1732430 13.01 MB/sec execute 599 sec latency 6.953 ms 2024-03-14T21:27:20.462 INFO:tasks.workunit.client.0.smithi163.stdout: 1 cleanup 600 sec 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: 0 cleanup 600 sec 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: Operation Count AvgLat MaxLat 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: ---------------------------------------- 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: NTCreateX 248596 0.861 125.906 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: Close 182589 0.008 1.242 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: Rename 10523 3.755 127.660 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: Unlink 50214 1.727 84.317 2024-03-14T21:27:20.597 INFO:tasks.workunit.client.0.smithi163.stdout: Deltree 6 72.084 89.093 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: Mkdir 3 0.041 0.042 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: Qpathinfo 225353 0.558 220.761 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: Qfileinfo 39480 0.005 0.098 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: Qfsinfo 41313 0.516 42.520 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: Sfileinfo 20230 0.073 0.261 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: Find 87120 0.279 66.250 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: WriteX 123874 0.066 0.768 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: ReadX 389709 0.011 4.226 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: LockX 810 0.728 3.466 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: UnlockX 810 0.659 3.758 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: Flush 17409 3.438 120.567 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:27:20.598 INFO:tasks.workunit.client.0.smithi163.stdout:Throughput 13.0111 MB/sec 1 clients 1 procs max_latency=220.771 ms 2024-03-14T21:27:20.601 INFO:tasks.workunit.client.0.smithi163.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-14T21:27:20.601 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:27:20.601 INFO:tasks.workunit.client.0.smithi163.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-14T21:27:20.604 INFO:tasks.workunit.client.0.smithi163.stdout:9 of 10 processes prepared for launch 0 sec 2024-03-14T21:27:20.704 INFO:tasks.workunit.client.0.smithi163.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-14T21:27:20.705 INFO:tasks.workunit.client.0.smithi163.stdout:releasing clients 2024-03-14T21:27:21.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 476 166.72 MB/sec warmup 1 sec latency 28.970 ms 2024-03-14T21:27:22.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 1106 114.76 MB/sec warmup 2 sec latency 49.442 ms 2024-03-14T21:27:23.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2109 89.07 MB/sec warmup 3 sec latency 38.961 ms 2024-03-14T21:27:24.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2927 83.38 MB/sec warmup 4 sec latency 36.985 ms 2024-03-14T21:27:25.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 3523 76.23 MB/sec warmup 5 sec latency 159.933 ms 2024-03-14T21:27:26.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 3860 66.32 MB/sec warmup 6 sec latency 82.294 ms 2024-03-14T21:27:27.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4104 57.26 MB/sec warmup 7 sec latency 142.641 ms 2024-03-14T21:27:28.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4459 51.08 MB/sec warmup 8 sec latency 137.718 ms 2024-03-14T21:27:29.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 5320 50.46 MB/sec warmup 9 sec latency 41.073 ms 2024-03-14T21:27:30.706 INFO:tasks.workunit.client.0.smithi163.stdout: 10 6082 48.19 MB/sec warmup 10 sec latency 117.683 ms 2024-03-14T21:27:31.706 INFO:tasks.workunit.client.0.smithi163.stdout: 10 6989 50.61 MB/sec warmup 11 sec latency 33.345 ms 2024-03-14T21:27:32.706 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7397 49.12 MB/sec warmup 12 sec latency 75.730 ms 2024-03-14T21:27:33.706 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7607 45.55 MB/sec warmup 13 sec latency 101.235 ms 2024-03-14T21:27:34.706 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7889 42.71 MB/sec warmup 14 sec latency 118.540 ms 2024-03-14T21:27:35.706 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8338 41.42 MB/sec warmup 15 sec latency 126.897 ms 2024-03-14T21:27:36.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 9191 40.76 MB/sec warmup 16 sec latency 42.644 ms 2024-03-14T21:27:37.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10073 42.12 MB/sec warmup 17 sec latency 33.895 ms 2024-03-14T21:27:38.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10777 42.89 MB/sec warmup 18 sec latency 67.016 ms 2024-03-14T21:27:39.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 11039 41.18 MB/sec warmup 19 sec latency 67.363 ms 2024-03-14T21:27:40.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 11237 39.21 MB/sec warmup 20 sec latency 119.822 ms 2024-03-14T21:27:41.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 11562 37.74 MB/sec warmup 21 sec latency 146.383 ms 2024-03-14T21:27:42.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12316 37.98 MB/sec warmup 22 sec latency 48.617 ms 2024-03-14T21:27:43.707 INFO:tasks.workunit.client.0.smithi163.stdout: 10 13252 37.91 MB/sec warmup 23 sec latency 36.075 ms 2024-03-14T21:27:44.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 13928 38.84 MB/sec warmup 24 sec latency 60.229 ms 2024-03-14T21:27:45.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14388 38.67 MB/sec warmup 25 sec latency 139.705 ms 2024-03-14T21:27:46.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14566 37.42 MB/sec warmup 26 sec latency 129.258 ms 2024-03-14T21:27:47.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14777 36.13 MB/sec warmup 27 sec latency 116.710 ms 2024-03-14T21:27:48.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15119 35.24 MB/sec warmup 28 sec latency 116.476 ms 2024-03-14T21:27:49.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15756 34.99 MB/sec warmup 29 sec latency 51.014 ms 2024-03-14T21:27:50.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16476 34.83 MB/sec warmup 30 sec latency 132.026 ms 2024-03-14T21:27:51.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 17200 35.37 MB/sec warmup 31 sec latency 56.162 ms 2024-03-14T21:27:52.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 17838 35.94 MB/sec warmup 32 sec latency 67.978 ms 2024-03-14T21:27:53.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18106 35.28 MB/sec warmup 33 sec latency 74.803 ms 2024-03-14T21:27:54.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18319 34.32 MB/sec warmup 34 sec latency 91.684 ms 2024-03-14T21:27:55.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18622 33.58 MB/sec warmup 35 sec latency 116.685 ms 2024-03-14T21:27:56.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18714 32.79 MB/sec warmup 36 sec latency 402.699 ms 2024-03-14T21:27:57.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 19242 32.58 MB/sec warmup 37 sec latency 126.730 ms 2024-03-14T21:27:58.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 19980 32.44 MB/sec warmup 38 sec latency 78.446 ms 2024-03-14T21:27:59.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20766 33.01 MB/sec warmup 39 sec latency 32.496 ms 2024-03-14T21:28:00.709 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21399 33.51 MB/sec warmup 40 sec latency 70.558 ms 2024-03-14T21:28:01.710 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21662 33.06 MB/sec warmup 41 sec latency 123.532 ms 2024-03-14T21:28:02.710 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21794 32.31 MB/sec warmup 42 sec latency 347.067 ms 2024-03-14T21:28:03.710 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22029 31.65 MB/sec warmup 43 sec latency 166.451 ms 2024-03-14T21:28:04.710 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22433 31.47 MB/sec warmup 44 sec latency 76.331 ms 2024-03-14T21:28:05.710 INFO:tasks.workunit.client.0.smithi163.stdout: 10 23209 31.32 MB/sec warmup 45 sec latency 87.674 ms 2024-03-14T21:28:06.710 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24013 31.61 MB/sec warmup 46 sec latency 40.746 ms 2024-03-14T21:28:07.710 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24526 31.89 MB/sec warmup 47 sec latency 262.927 ms 2024-03-14T21:28:08.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 25066 32.08 MB/sec warmup 48 sec latency 74.734 ms 2024-03-14T21:28:09.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 25316 31.59 MB/sec warmup 49 sec latency 57.887 ms 2024-03-14T21:28:10.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 25598 31.05 MB/sec warmup 50 sec latency 120.323 ms 2024-03-14T21:28:11.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26012 30.81 MB/sec warmup 51 sec latency 92.148 ms 2024-03-14T21:28:12.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26585 30.73 MB/sec warmup 52 sec latency 143.476 ms 2024-03-14T21:28:13.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27377 30.83 MB/sec warmup 53 sec latency 65.671 ms 2024-03-14T21:28:14.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28228 31.52 MB/sec warmup 54 sec latency 30.400 ms 2024-03-14T21:28:15.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28668 31.53 MB/sec warmup 55 sec latency 61.553 ms 2024-03-14T21:28:16.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28882 31.06 MB/sec warmup 56 sec latency 67.300 ms 2024-03-14T21:28:17.711 INFO:tasks.workunit.client.0.smithi163.stdout: 10 29081 30.57 MB/sec warmup 57 sec latency 229.945 ms 2024-03-14T21:28:18.712 INFO:tasks.workunit.client.0.smithi163.stdout: 10 29469 30.36 MB/sec warmup 58 sec latency 374.635 ms 2024-03-14T21:28:19.712 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30193 30.29 MB/sec warmup 59 sec latency 70.719 ms 2024-03-14T21:28:20.712 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31145 30.64 MB/sec warmup 60 sec latency 39.750 ms 2024-03-14T21:28:21.712 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31901 31.18 MB/sec warmup 61 sec latency 56.646 ms 2024-03-14T21:28:22.712 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32306 31.12 MB/sec warmup 62 sec latency 61.969 ms 2024-03-14T21:28:23.712 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32415 30.63 MB/sec warmup 63 sec latency 291.314 ms 2024-03-14T21:28:24.712 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32666 30.23 MB/sec warmup 64 sec latency 161.924 ms 2024-03-14T21:28:25.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 33116 30.13 MB/sec warmup 65 sec latency 47.884 ms 2024-03-14T21:28:26.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 33819 30.04 MB/sec warmup 66 sec latency 59.442 ms 2024-03-14T21:28:27.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34641 30.26 MB/sec warmup 67 sec latency 35.999 ms 2024-03-14T21:28:28.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 35368 30.71 MB/sec warmup 68 sec latency 32.273 ms 2024-03-14T21:28:29.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 35642 30.50 MB/sec warmup 69 sec latency 313.747 ms 2024-03-14T21:28:30.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 35915 30.30 MB/sec warmup 70 sec latency 115.423 ms 2024-03-14T21:28:31.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 36181 29.94 MB/sec warmup 71 sec latency 117.321 ms 2024-03-14T21:28:32.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 36574 29.84 MB/sec warmup 72 sec latency 62.067 ms 2024-03-14T21:28:33.713 INFO:tasks.workunit.client.0.smithi163.stdout: 10 37254 29.70 MB/sec warmup 73 sec latency 129.821 ms 2024-03-14T21:28:34.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 37813 29.76 MB/sec warmup 74 sec latency 157.339 ms 2024-03-14T21:28:35.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 38570 29.99 MB/sec warmup 75 sec latency 58.995 ms 2024-03-14T21:28:36.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 39153 30.16 MB/sec warmup 76 sec latency 35.887 ms 2024-03-14T21:28:37.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 39480 30.02 MB/sec warmup 77 sec latency 69.972 ms 2024-03-14T21:28:38.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 39774 29.73 MB/sec warmup 78 sec latency 111.050 ms 2024-03-14T21:28:39.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 40137 29.61 MB/sec warmup 79 sec latency 183.061 ms 2024-03-14T21:28:40.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 40636 29.36 MB/sec warmup 80 sec latency 435.316 ms 2024-03-14T21:28:41.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 41296 29.58 MB/sec warmup 81 sec latency 55.167 ms 2024-03-14T21:28:42.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 42165 29.79 MB/sec warmup 82 sec latency 33.271 ms 2024-03-14T21:28:43.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 42696 29.94 MB/sec warmup 83 sec latency 67.518 ms 2024-03-14T21:28:44.714 INFO:tasks.workunit.client.0.smithi163.stdout: 10 42999 29.78 MB/sec warmup 84 sec latency 90.947 ms 2024-03-14T21:28:45.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 43236 29.50 MB/sec warmup 85 sec latency 325.175 ms 2024-03-14T21:28:46.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 43610 29.42 MB/sec warmup 86 sec latency 60.962 ms 2024-03-14T21:28:47.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 44207 29.21 MB/sec warmup 87 sec latency 102.516 ms 2024-03-14T21:28:48.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 44912 29.47 MB/sec warmup 88 sec latency 45.468 ms 2024-03-14T21:28:49.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 45835 29.77 MB/sec warmup 89 sec latency 30.290 ms 2024-03-14T21:28:50.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 46196 29.69 MB/sec warmup 90 sec latency 205.795 ms 2024-03-14T21:28:51.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 46519 29.59 MB/sec warmup 91 sec latency 75.411 ms 2024-03-14T21:28:52.715 INFO:tasks.workunit.client.0.smithi163.stdout: 10 46866 29.39 MB/sec warmup 92 sec latency 100.430 ms 2024-03-14T21:28:53.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 47395 29.28 MB/sec warmup 93 sec latency 40.615 ms 2024-03-14T21:28:54.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 48070 29.27 MB/sec warmup 94 sec latency 82.961 ms 2024-03-14T21:28:55.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 48854 29.44 MB/sec warmup 95 sec latency 161.464 ms 2024-03-14T21:28:56.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 49408 29.61 MB/sec warmup 96 sec latency 69.550 ms 2024-03-14T21:28:57.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 49894 29.64 MB/sec warmup 97 sec latency 61.793 ms 2024-03-14T21:28:58.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 50223 29.46 MB/sec warmup 98 sec latency 112.386 ms 2024-03-14T21:28:59.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 50673 29.39 MB/sec warmup 99 sec latency 56.166 ms 2024-03-14T21:29:00.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 51198 29.21 MB/sec warmup 100 sec latency 123.754 ms 2024-03-14T21:29:01.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 51705 29.25 MB/sec warmup 101 sec latency 317.473 ms 2024-03-14T21:29:02.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 52526 29.40 MB/sec warmup 102 sec latency 62.385 ms 2024-03-14T21:29:03.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 53141 29.56 MB/sec warmup 103 sec latency 35.748 ms 2024-03-14T21:29:04.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 53566 29.55 MB/sec warmup 104 sec latency 58.400 ms 2024-03-14T21:29:05.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 53902 29.37 MB/sec warmup 105 sec latency 68.763 ms 2024-03-14T21:29:06.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 54145 29.21 MB/sec warmup 106 sec latency 314.794 ms 2024-03-14T21:29:07.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 54608 29.03 MB/sec warmup 107 sec latency 133.536 ms 2024-03-14T21:29:08.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 55395 29.22 MB/sec warmup 108 sec latency 37.184 ms 2024-03-14T21:29:09.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 56223 29.35 MB/sec warmup 109 sec latency 35.664 ms 2024-03-14T21:29:10.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 56828 29.48 MB/sec warmup 110 sec latency 57.918 ms 2024-03-14T21:29:11.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 57183 29.44 MB/sec warmup 111 sec latency 57.466 ms 2024-03-14T21:29:12.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 57367 29.21 MB/sec warmup 112 sec latency 271.781 ms 2024-03-14T21:29:13.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 57765 29.12 MB/sec warmup 113 sec latency 72.306 ms 2024-03-14T21:29:14.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 58408 29.03 MB/sec warmup 114 sec latency 95.051 ms 2024-03-14T21:29:15.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 59320 29.21 MB/sec warmup 115 sec latency 32.415 ms 2024-03-14T21:29:16.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 60042 29.43 MB/sec warmup 116 sec latency 45.569 ms 2024-03-14T21:29:17.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 60434 29.43 MB/sec warmup 117 sec latency 241.847 ms 2024-03-14T21:29:18.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 60786 29.32 MB/sec warmup 118 sec latency 105.929 ms 2024-03-14T21:29:19.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 61117 29.18 MB/sec warmup 119 sec latency 78.346 ms 2024-03-14T21:29:21.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 62297 35.91 MB/sec execute 1 sec latency 90.512 ms 2024-03-14T21:29:22.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:29:22.583 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:29:22.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:29:22.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 62933 34.73 MB/sec execute 2 sec latency 277.445 ms 2024-03-14T21:29:23.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 63509 36.36 MB/sec execute 3 sec latency 66.505 ms 2024-03-14T21:29:24.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 64078 37.69 MB/sec execute 4 sec latency 58.033 ms 2024-03-14T21:29:25.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 64378 32.90 MB/sec execute 5 sec latency 59.020 ms 2024-03-14T21:29:26.719 INFO:tasks.workunit.client.0.smithi163.stdout: 10 64752 29.54 MB/sec execute 6 sec latency 94.828 ms 2024-03-14T21:29:27.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 65142 27.41 MB/sec execute 7 sec latency 299.399 ms 2024-03-14T21:29:28.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 65775 27.57 MB/sec execute 8 sec latency 110.188 ms 2024-03-14T21:29:29.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 66710 30.71 MB/sec execute 9 sec latency 32.324 ms 2024-03-14T21:29:30.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 67361 32.55 MB/sec execute 10 sec latency 45.779 ms 2024-03-14T21:29:31.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 67775 31.64 MB/sec execute 11 sec latency 94.112 ms 2024-03-14T21:29:32.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 67993 29.70 MB/sec execute 12 sec latency 289.772 ms 2024-03-14T21:29:33.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 68348 28.47 MB/sec execute 13 sec latency 88.436 ms 2024-03-14T21:29:34.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 68985 28.32 MB/sec execute 14 sec latency 79.988 ms 2024-03-14T21:29:35.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 69788 29.19 MB/sec execute 15 sec latency 36.994 ms 2024-03-14T21:29:36.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 70596 30.60 MB/sec execute 16 sec latency 52.377 ms 2024-03-14T21:29:37.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 71000 30.77 MB/sec execute 17 sec latency 194.536 ms 2024-03-14T21:29:38.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 71345 29.99 MB/sec execute 18 sec latency 125.483 ms 2024-03-14T21:29:39.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 71680 29.13 MB/sec execute 19 sec latency 96.091 ms 2024-03-14T21:29:40.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 72233 28.77 MB/sec execute 20 sec latency 42.933 ms 2024-03-14T21:29:41.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 72872 28.68 MB/sec execute 21 sec latency 160.494 ms 2024-03-14T21:29:42.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 73641 29.70 MB/sec execute 22 sec latency 157.997 ms 2024-03-14T21:29:43.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 74076 29.50 MB/sec execute 23 sec latency 419.727 ms 2024-03-14T21:29:44.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 74548 29.60 MB/sec execute 24 sec latency 77.606 ms 2024-03-14T21:29:45.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 74920 29.18 MB/sec execute 25 sec latency 65.965 ms 2024-03-14T21:29:46.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 75252 28.52 MB/sec execute 26 sec latency 76.866 ms 2024-03-14T21:29:47.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 75876 28.55 MB/sec execute 27 sec latency 38.658 ms 2024-03-14T21:29:48.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 76532 28.39 MB/sec execute 28 sec latency 101.519 ms 2024-03-14T21:29:49.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 76953 28.39 MB/sec execute 29 sec latency 269.803 ms 2024-03-14T21:29:50.722 INFO:tasks.workunit.client.0.smithi163.stdout: 10 77801 29.38 MB/sec execute 30 sec latency 40.540 ms 2024-03-14T21:29:51.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 78204 29.34 MB/sec execute 31 sec latency 71.985 ms 2024-03-14T21:29:52.710 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:29:52.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:29:52.715 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:29:52.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 78545 28.94 MB/sec execute 32 sec latency 134.730 ms 2024-03-14T21:29:52.751 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-03-14T21:29:52.771+0000 7f5aad84b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46338) UID: 0 2024-03-14T21:29:52.751 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T21:29:52.771+0000 7f137ed27640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46338) UID: 0 2024-03-14T21:29:53.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 78858 28.38 MB/sec execute 33 sec latency 107.464 ms 2024-03-14T21:29:54.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 79173 28.05 MB/sec execute 34 sec latency 383.470 ms 2024-03-14T21:29:55.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 79732 27.79 MB/sec execute 35 sec latency 116.941 ms 2024-03-14T21:29:56.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 80345 27.94 MB/sec execute 36 sec latency 105.658 ms 2024-03-14T21:29:57.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 81115 28.52 MB/sec execute 37 sec latency 34.165 ms 2024-03-14T21:29:58.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 81603 28.78 MB/sec execute 38 sec latency 57.793 ms 2024-03-14T21:29:59.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 81878 28.40 MB/sec execute 39 sec latency 301.247 ms 2024-03-14T21:30:00.723 INFO:tasks.workunit.client.0.smithi163.stdout: 10 82184 28.01 MB/sec execute 40 sec latency 300.641 ms 2024-03-14T21:30:01.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 82557 27.65 MB/sec execute 41 sec latency 83.429 ms 2024-03-14T21:30:02.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 83212 27.60 MB/sec execute 42 sec latency 71.703 ms 2024-03-14T21:30:03.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 83903 27.87 MB/sec execute 43 sec latency 40.764 ms 2024-03-14T21:30:04.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 84618 28.26 MB/sec execute 44 sec latency 209.654 ms 2024-03-14T21:30:05.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 85083 28.34 MB/sec execute 45 sec latency 242.456 ms 2024-03-14T21:30:06.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 85443 28.20 MB/sec execute 46 sec latency 140.770 ms 2024-03-14T21:30:07.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 85782 27.95 MB/sec execute 47 sec latency 93.785 ms 2024-03-14T21:30:08.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 86187 27.73 MB/sec execute 48 sec latency 81.949 ms 2024-03-14T21:30:09.724 INFO:tasks.workunit.client.0.smithi163.stdout: 10 86750 27.60 MB/sec execute 49 sec latency 132.724 ms 2024-03-14T21:30:10.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 87219 27.56 MB/sec execute 50 sec latency 196.760 ms 2024-03-14T21:30:11.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 87890 27.86 MB/sec execute 51 sec latency 60.776 ms 2024-03-14T21:30:12.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 88549 28.11 MB/sec execute 52 sec latency 69.732 ms 2024-03-14T21:30:13.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 88915 27.99 MB/sec execute 53 sec latency 101.060 ms 2024-03-14T21:30:14.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 89313 27.84 MB/sec execute 54 sec latency 91.551 ms 2024-03-14T21:30:15.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 89583 27.61 MB/sec execute 55 sec latency 341.782 ms 2024-03-14T21:30:16.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 90140 27.48 MB/sec execute 56 sec latency 120.429 ms 2024-03-14T21:30:17.725 INFO:tasks.workunit.client.0.smithi163.stdout: 10 90727 27.47 MB/sec execute 57 sec latency 114.061 ms 2024-03-14T21:30:18.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 91487 27.88 MB/sec execute 58 sec latency 37.051 ms 2024-03-14T21:30:19.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 92062 27.97 MB/sec execute 59 sec latency 79.118 ms 2024-03-14T21:30:20.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 92441 27.92 MB/sec execute 60 sec latency 34.980 ms 2024-03-14T21:30:21.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 92623 27.57 MB/sec execute 61 sec latency 405.589 ms 2024-03-14T21:30:22.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 92996 27.44 MB/sec execute 62 sec latency 106.725 ms 2024-03-14T21:30:23.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 93515 27.36 MB/sec execute 63 sec latency 54.064 ms 2024-03-14T21:30:24.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 94112 27.25 MB/sec execute 64 sec latency 55.680 ms 2024-03-14T21:30:25.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 94782 27.54 MB/sec execute 65 sec latency 44.725 ms 2024-03-14T21:30:26.726 INFO:tasks.workunit.client.0.smithi163.stdout: 10 95309 27.66 MB/sec execute 66 sec latency 210.680 ms 2024-03-14T21:30:27.727 INFO:tasks.workunit.client.0.smithi163.stdout: 10 95709 27.58 MB/sec execute 67 sec latency 274.144 ms 2024-03-14T21:30:28.727 INFO:tasks.workunit.client.0.smithi163.stdout: 10 96026 27.48 MB/sec execute 68 sec latency 138.413 ms 2024-03-14T21:30:29.727 INFO:tasks.workunit.client.0.smithi163.stdout: 10 96356 27.30 MB/sec execute 69 sec latency 99.358 ms 2024-03-14T21:30:30.727 INFO:tasks.workunit.client.0.smithi163.stdout: 10 96857 27.23 MB/sec execute 70 sec latency 102.295 ms 2024-03-14T21:30:31.728 INFO:tasks.workunit.client.0.smithi163.stdout: 10 97451 27.10 MB/sec execute 71 sec latency 137.461 ms 2024-03-14T21:30:32.729 INFO:tasks.workunit.client.0.smithi163.stdout: 10 97992 27.20 MB/sec execute 72 sec latency 226.805 ms 2024-03-14T21:30:33.729 INFO:tasks.workunit.client.0.smithi163.stdout: 10 98787 27.49 MB/sec execute 73 sec latency 77.228 ms 2024-03-14T21:30:34.729 INFO:tasks.workunit.client.0.smithi163.stdout: 10 99308 27.61 MB/sec execute 74 sec latency 67.021 ms 2024-03-14T21:30:35.729 INFO:tasks.workunit.client.0.smithi163.stdout: 10 99720 27.57 MB/sec execute 75 sec latency 67.354 ms 2024-03-14T21:30:36.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 100037 27.38 MB/sec execute 76 sec latency 88.065 ms 2024-03-14T21:30:37.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 100286 27.21 MB/sec execute 77 sec latency 308.650 ms 2024-03-14T21:30:38.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 100706 27.03 MB/sec execute 78 sec latency 128.237 ms 2024-03-14T21:30:39.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 101272 26.95 MB/sec execute 79 sec latency 51.795 ms 2024-03-14T21:30:40.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 101960 27.22 MB/sec execute 80 sec latency 41.190 ms 2024-03-14T21:30:41.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 102588 27.37 MB/sec execute 81 sec latency 74.248 ms 2024-03-14T21:30:42.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 102866 27.27 MB/sec execute 82 sec latency 359.819 ms 2024-03-14T21:30:43.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 103218 27.17 MB/sec execute 83 sec latency 370.450 ms 2024-03-14T21:30:44.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 103551 27.01 MB/sec execute 84 sec latency 122.843 ms 2024-03-14T21:30:45.730 INFO:tasks.workunit.client.0.smithi163.stdout: 10 104067 27.03 MB/sec execute 85 sec latency 42.103 ms 2024-03-14T21:30:46.731 INFO:tasks.workunit.client.0.smithi163.stdout: 10 104705 26.92 MB/sec execute 86 sec latency 66.353 ms 2024-03-14T21:30:47.731 INFO:tasks.workunit.client.0.smithi163.stdout: 10 105164 27.03 MB/sec execute 87 sec latency 198.748 ms 2024-03-14T21:30:48.731 INFO:tasks.workunit.client.0.smithi163.stdout: 10 105842 27.20 MB/sec execute 88 sec latency 298.929 ms 2024-03-14T21:30:49.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 106291 27.18 MB/sec execute 89 sec latency 62.884 ms 2024-03-14T21:30:50.731 INFO:tasks.workunit.client.0.smithi163.stdout: 10 106665 27.14 MB/sec execute 90 sec latency 67.711 ms 2024-03-14T21:30:51.732 INFO:tasks.workunit.client.0.smithi163.stdout: 10 106993 27.00 MB/sec execute 91 sec latency 109.203 ms 2024-03-14T21:30:52.731 INFO:tasks.workunit.client.0.smithi163.stdout: 10 107455 26.95 MB/sec execute 92 sec latency 67.351 ms 2024-03-14T21:30:53.732 INFO:tasks.workunit.client.0.smithi163.stdout: 10 107899 26.83 MB/sec execute 93 sec latency 272.177 ms 2024-03-14T21:30:54.732 INFO:tasks.workunit.client.0.smithi163.stdout: 10 108380 26.72 MB/sec execute 94 sec latency 89.405 ms 2024-03-14T21:30:55.732 INFO:tasks.workunit.client.0.smithi163.stdout: 10 109210 27.02 MB/sec execute 95 sec latency 46.739 ms 2024-03-14T21:30:56.732 INFO:tasks.workunit.client.0.smithi163.stdout: 10 109843 27.22 MB/sec execute 96 sec latency 39.151 ms 2024-03-14T21:30:57.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 110220 27.19 MB/sec execute 97 sec latency 79.991 ms 2024-03-14T21:30:58.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 110573 27.05 MB/sec execute 98 sec latency 95.075 ms 2024-03-14T21:30:59.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 110826 26.90 MB/sec execute 99 sec latency 351.224 ms 2024-03-14T21:31:00.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 111297 26.80 MB/sec execute 100 sec latency 76.934 ms 2024-03-14T21:31:01.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 111907 26.78 MB/sec execute 101 sec latency 105.814 ms 2024-03-14T21:31:02.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 112626 26.99 MB/sec execute 102 sec latency 40.292 ms 2024-03-14T21:31:03.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 113262 27.16 MB/sec execute 103 sec latency 54.291 ms 2024-03-14T21:31:04.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 113585 27.05 MB/sec execute 104 sec latency 294.918 ms 2024-03-14T21:31:05.733 INFO:tasks.workunit.client.0.smithi163.stdout: 10 113953 27.00 MB/sec execute 105 sec latency 127.526 ms 2024-03-14T21:31:06.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 114360 26.92 MB/sec execute 106 sec latency 94.080 ms 2024-03-14T21:31:07.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 114898 26.85 MB/sec execute 107 sec latency 53.425 ms 2024-03-14T21:31:08.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 115545 26.86 MB/sec execute 108 sec latency 75.183 ms 2024-03-14T21:31:09.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 116125 26.97 MB/sec execute 109 sec latency 124.757 ms 2024-03-14T21:31:10.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 116728 27.09 MB/sec execute 110 sec latency 73.695 ms 2024-03-14T21:31:11.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 117201 27.12 MB/sec execute 111 sec latency 36.809 ms 2024-03-14T21:31:12.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 117539 27.02 MB/sec execute 112 sec latency 92.833 ms 2024-03-14T21:31:13.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 117907 26.96 MB/sec execute 113 sec latency 115.451 ms 2024-03-14T21:31:14.734 INFO:tasks.workunit.client.0.smithi163.stdout: 10 118334 26.86 MB/sec execute 114 sec latency 232.974 ms 2024-03-14T21:31:15.735 INFO:tasks.workunit.client.0.smithi163.stdout: 10 118869 26.80 MB/sec execute 115 sec latency 105.376 ms 2024-03-14T21:31:16.735 INFO:tasks.workunit.client.0.smithi163.stdout: 10 119535 26.93 MB/sec execute 116 sec latency 48.150 ms 2024-03-14T21:31:17.735 INFO:tasks.workunit.client.0.smithi163.stdout: 10 120185 27.04 MB/sec execute 117 sec latency 42.043 ms 2024-03-14T21:31:18.735 INFO:tasks.workunit.client.0.smithi163.stdout: 10 120592 27.04 MB/sec execute 118 sec latency 120.997 ms 2024-03-14T21:31:19.735 INFO:tasks.workunit.client.0.smithi163.stdout: 10 121029 27.03 MB/sec execute 119 sec latency 41.909 ms 2024-03-14T21:31:20.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 121229 26.87 MB/sec execute 120 sec latency 408.432 ms 2024-03-14T21:31:21.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 121641 26.85 MB/sec execute 121 sec latency 93.751 ms 2024-03-14T21:31:22.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 122247 26.76 MB/sec execute 122 sec latency 67.214 ms 2024-03-14T21:31:23.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 122814 26.83 MB/sec execute 123 sec latency 70.780 ms 2024-03-14T21:31:24.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 123593 27.01 MB/sec execute 124 sec latency 41.343 ms 2024-03-14T21:31:25.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 123924 26.95 MB/sec execute 125 sec latency 247.604 ms 2024-03-14T21:31:26.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 124233 26.87 MB/sec execute 126 sec latency 88.626 ms 2024-03-14T21:31:27.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 124649 26.85 MB/sec execute 127 sec latency 49.007 ms 2024-03-14T21:31:28.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 125035 26.80 MB/sec execute 128 sec latency 111.583 ms 2024-03-14T21:31:29.736 INFO:tasks.workunit.client.0.smithi163.stdout: 10 125622 26.72 MB/sec execute 129 sec latency 59.952 ms 2024-03-14T21:31:30.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 126071 26.68 MB/sec execute 130 sec latency 212.189 ms 2024-03-14T21:31:31.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 126783 26.83 MB/sec execute 131 sec latency 107.068 ms 2024-03-14T21:31:32.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 127443 26.95 MB/sec execute 132 sec latency 49.346 ms 2024-03-14T21:31:33.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 127883 26.98 MB/sec execute 133 sec latency 67.387 ms 2024-03-14T21:31:34.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 128197 26.87 MB/sec execute 134 sec latency 154.125 ms 2024-03-14T21:31:35.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 128595 26.84 MB/sec execute 135 sec latency 115.858 ms 2024-03-14T21:31:36.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 128861 26.71 MB/sec execute 136 sec latency 260.699 ms 2024-03-14T21:31:37.737 INFO:tasks.workunit.client.0.smithi163.stdout: 10 129420 26.64 MB/sec execute 137 sec latency 88.293 ms 2024-03-14T21:31:38.738 INFO:tasks.workunit.client.0.smithi163.stdout: 10 130101 26.76 MB/sec execute 138 sec latency 102.336 ms 2024-03-14T21:31:39.738 INFO:tasks.workunit.client.0.smithi163.stdout: 10 130832 26.89 MB/sec execute 139 sec latency 35.852 ms 2024-03-14T21:31:40.738 INFO:tasks.workunit.client.0.smithi163.stdout: 10 131228 26.90 MB/sec execute 140 sec latency 69.396 ms 2024-03-14T21:31:41.738 INFO:tasks.workunit.client.0.smithi163.stdout: 10 131457 26.79 MB/sec execute 141 sec latency 391.766 ms 2024-03-14T21:31:42.738 INFO:tasks.workunit.client.0.smithi163.stdout: 10 131781 26.72 MB/sec execute 142 sec latency 100.878 ms 2024-03-14T21:31:43.738 INFO:tasks.workunit.client.0.smithi163.stdout: 10 132207 26.70 MB/sec execute 143 sec latency 62.797 ms 2024-03-14T21:31:44.738 INFO:tasks.workunit.client.0.smithi163.stdout: 10 132801 26.62 MB/sec execute 144 sec latency 84.608 ms 2024-03-14T21:31:45.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 133374 26.68 MB/sec execute 145 sec latency 62.856 ms 2024-03-14T21:31:46.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 134175 26.81 MB/sec execute 146 sec latency 41.706 ms 2024-03-14T21:31:47.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 134529 26.80 MB/sec execute 147 sec latency 325.635 ms 2024-03-14T21:31:48.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 134914 26.80 MB/sec execute 148 sec latency 88.029 ms 2024-03-14T21:31:49.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 135292 26.74 MB/sec execute 149 sec latency 68.413 ms 2024-03-14T21:31:50.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 135658 26.67 MB/sec execute 150 sec latency 53.809 ms 2024-03-14T21:31:51.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 136118 26.61 MB/sec execute 151 sec latency 94.631 ms 2024-03-14T21:31:52.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 136562 26.55 MB/sec execute 152 sec latency 162.126 ms 2024-03-14T21:31:53.739 INFO:tasks.workunit.client.0.smithi163.stdout: 10 137071 26.59 MB/sec execute 153 sec latency 287.706 ms 2024-03-14T21:31:54.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 137927 26.75 MB/sec execute 154 sec latency 33.993 ms 2024-03-14T21:31:55.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 138387 26.81 MB/sec execute 155 sec latency 69.697 ms 2024-03-14T21:31:56.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 138767 26.76 MB/sec execute 156 sec latency 80.203 ms 2024-03-14T21:31:57.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 139143 26.68 MB/sec execute 157 sec latency 72.992 ms 2024-03-14T21:31:58.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 139521 26.63 MB/sec execute 158 sec latency 282.721 ms 2024-03-14T21:31:59.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 140067 26.57 MB/sec execute 159 sec latency 82.033 ms 2024-03-14T21:32:00.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 140815 26.71 MB/sec execute 160 sec latency 40.946 ms 2024-03-14T21:32:01.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 141528 26.80 MB/sec execute 161 sec latency 40.688 ms 2024-03-14T21:32:02.740 INFO:tasks.workunit.client.0.smithi163.stdout: 10 141966 26.84 MB/sec execute 162 sec latency 73.273 ms 2024-03-14T21:32:03.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 142309 26.79 MB/sec execute 163 sec latency 103.774 ms 2024-03-14T21:32:04.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 142484 26.67 MB/sec execute 164 sec latency 273.489 ms 2024-03-14T21:32:05.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 142801 26.58 MB/sec execute 165 sec latency 100.840 ms 2024-03-14T21:32:06.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 143301 26.52 MB/sec execute 166 sec latency 55.277 ms 2024-03-14T21:32:07.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 143850 26.51 MB/sec execute 167 sec latency 65.904 ms 2024-03-14T21:32:08.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 144595 26.67 MB/sec execute 168 sec latency 35.923 ms 2024-03-14T21:32:09.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 145107 26.68 MB/sec execute 169 sec latency 292.239 ms 2024-03-14T21:32:10.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 145421 26.65 MB/sec execute 170 sec latency 314.982 ms 2024-03-14T21:32:11.741 INFO:tasks.workunit.client.0.smithi163.stdout: 10 145796 26.62 MB/sec execute 171 sec latency 122.301 ms 2024-03-14T21:32:12.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 146186 26.56 MB/sec execute 172 sec latency 76.000 ms 2024-03-14T21:32:13.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 146749 26.54 MB/sec execute 173 sec latency 49.242 ms 2024-03-14T21:32:14.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 147200 26.47 MB/sec execute 174 sec latency 174.372 ms 2024-03-14T21:32:15.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 147731 26.54 MB/sec execute 175 sec latency 257.569 ms 2024-03-14T21:32:16.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 148516 26.64 MB/sec execute 176 sec latency 62.798 ms 2024-03-14T21:32:17.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 148935 26.67 MB/sec execute 177 sec latency 69.352 ms 2024-03-14T21:32:18.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 149330 26.63 MB/sec execute 178 sec latency 120.744 ms 2024-03-14T21:32:19.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 149732 26.59 MB/sec execute 179 sec latency 64.664 ms 2024-03-14T21:32:20.742 INFO:tasks.workunit.client.0.smithi163.stdout: 10 150073 26.53 MB/sec execute 180 sec latency 344.046 ms 2024-03-14T21:32:21.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 150592 26.48 MB/sec execute 181 sec latency 130.463 ms 2024-03-14T21:32:22.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 151209 26.55 MB/sec execute 182 sec latency 86.279 ms 2024-03-14T21:32:23.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 152086 26.67 MB/sec execute 183 sec latency 31.918 ms 2024-03-14T21:32:24.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 152503 26.71 MB/sec execute 184 sec latency 82.276 ms 2024-03-14T21:32:25.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 152895 26.69 MB/sec execute 185 sec latency 105.801 ms 2024-03-14T21:32:26.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 153067 26.58 MB/sec execute 186 sec latency 196.074 ms 2024-03-14T21:32:27.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 153453 26.53 MB/sec execute 187 sec latency 140.993 ms 2024-03-14T21:32:28.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 154017 26.47 MB/sec execute 188 sec latency 103.946 ms 2024-03-14T21:32:29.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 154682 26.55 MB/sec execute 189 sec latency 50.483 ms 2024-03-14T21:32:30.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 155583 26.68 MB/sec execute 190 sec latency 44.898 ms 2024-03-14T21:32:31.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 155934 26.69 MB/sec execute 191 sec latency 240.970 ms 2024-03-14T21:32:32.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 156294 26.66 MB/sec execute 192 sec latency 281.158 ms 2024-03-14T21:32:33.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 156571 26.60 MB/sec execute 193 sec latency 141.303 ms 2024-03-14T21:32:34.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 156922 26.53 MB/sec execute 194 sec latency 59.231 ms 2024-03-14T21:32:35.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 157360 26.47 MB/sec execute 195 sec latency 102.761 ms 2024-03-14T21:32:36.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 157848 26.44 MB/sec execute 196 sec latency 151.565 ms 2024-03-14T21:32:37.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 158428 26.49 MB/sec execute 197 sec latency 224.651 ms 2024-03-14T21:32:38.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 159206 26.59 MB/sec execute 198 sec latency 39.484 ms 2024-03-14T21:32:39.744 INFO:tasks.workunit.client.0.smithi163.stdout: 10 159691 26.64 MB/sec execute 199 sec latency 55.503 ms 2024-03-14T21:32:40.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 160045 26.62 MB/sec execute 200 sec latency 110.933 ms 2024-03-14T21:32:41.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 160360 26.54 MB/sec execute 201 sec latency 100.115 ms 2024-03-14T21:32:42.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 160640 26.47 MB/sec execute 202 sec latency 287.086 ms 2024-03-14T21:32:43.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 161130 26.41 MB/sec execute 203 sec latency 186.495 ms 2024-03-14T21:32:44.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 161735 26.45 MB/sec execute 204 sec latency 45.956 ms 2024-03-14T21:32:45.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 162516 26.54 MB/sec execute 205 sec latency 37.463 ms 2024-03-14T21:32:46.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 163112 26.62 MB/sec execute 206 sec latency 77.726 ms 2024-03-14T21:32:47.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 163388 26.60 MB/sec execute 207 sec latency 435.995 ms 2024-03-14T21:32:48.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 163643 26.52 MB/sec execute 208 sec latency 417.687 ms 2024-03-14T21:32:49.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 163902 26.43 MB/sec execute 209 sec latency 119.896 ms 2024-03-14T21:32:50.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 164329 26.40 MB/sec execute 210 sec latency 90.372 ms 2024-03-14T21:32:51.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 165051 26.39 MB/sec execute 211 sec latency 39.049 ms 2024-03-14T21:32:52.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 165672 26.45 MB/sec execute 212 sec latency 252.705 ms 2024-03-14T21:32:53.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 166358 26.56 MB/sec execute 213 sec latency 269.688 ms 2024-03-14T21:32:54.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 166812 26.58 MB/sec execute 214 sec latency 51.236 ms 2024-03-14T21:32:55.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 167110 26.52 MB/sec execute 215 sec latency 100.859 ms 2024-03-14T21:32:56.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 167437 26.46 MB/sec execute 216 sec latency 108.106 ms 2024-03-14T21:32:57.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 167862 26.43 MB/sec execute 217 sec latency 134.187 ms 2024-03-14T21:32:58.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 168392 26.39 MB/sec execute 218 sec latency 225.180 ms 2024-03-14T21:32:59.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 169083 26.43 MB/sec execute 219 sec latency 95.206 ms 2024-03-14T21:33:00.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 169926 26.58 MB/sec execute 220 sec latency 34.649 ms 2024-03-14T21:33:01.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 170424 26.64 MB/sec execute 221 sec latency 65.042 ms 2024-03-14T21:33:02.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 170721 26.56 MB/sec execute 222 sec latency 93.450 ms 2024-03-14T21:33:03.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 171043 26.49 MB/sec execute 223 sec latency 117.291 ms 2024-03-14T21:33:04.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 171405 26.45 MB/sec execute 224 sec latency 324.912 ms 2024-03-14T21:33:05.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 172084 26.44 MB/sec execute 225 sec latency 66.290 ms 2024-03-14T21:33:06.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 172888 26.53 MB/sec execute 226 sec latency 82.003 ms 2024-03-14T21:33:07.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 173640 26.66 MB/sec execute 227 sec latency 50.060 ms 2024-03-14T21:33:08.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 174050 26.67 MB/sec execute 228 sec latency 71.160 ms 2024-03-14T21:33:09.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 174258 26.59 MB/sec execute 229 sec latency 299.230 ms 2024-03-14T21:33:10.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 174606 26.52 MB/sec execute 230 sec latency 186.969 ms 2024-03-14T21:33:11.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 175104 26.49 MB/sec execute 231 sec latency 107.184 ms 2024-03-14T21:33:12.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 175925 26.53 MB/sec execute 232 sec latency 52.587 ms 2024-03-14T21:33:13.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 176737 26.65 MB/sec execute 233 sec latency 35.705 ms 2024-03-14T21:33:14.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 177274 26.72 MB/sec execute 234 sec latency 193.452 ms 2024-03-14T21:33:15.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 177679 26.71 MB/sec execute 235 sec latency 206.579 ms 2024-03-14T21:33:16.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 177901 26.62 MB/sec execute 236 sec latency 155.981 ms 2024-03-14T21:33:17.749 INFO:tasks.workunit.client.0.smithi163.stdout: 10 178264 26.56 MB/sec execute 237 sec latency 64.630 ms 2024-03-14T21:33:18.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 178839 26.56 MB/sec execute 238 sec latency 76.874 ms 2024-03-14T21:33:19.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 179533 26.57 MB/sec execute 239 sec latency 149.373 ms 2024-03-14T21:33:20.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 180264 26.65 MB/sec execute 240 sec latency 247.007 ms 2024-03-14T21:33:20.926 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:33:20.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:33:20.933 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:33:21.041 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-03-14T21:33:21.062+0000 7f5aad84b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46380) UID: 0 2024-03-14T21:33:21.042 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T21:33:21.062+0000 7f137ed27640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46380) UID: 0 2024-03-14T21:33:21.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 180845 26.75 MB/sec execute 241 sec latency 47.673 ms 2024-03-14T21:33:22.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 181202 26.73 MB/sec execute 242 sec latency 60.581 ms 2024-03-14T21:33:23.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 181453 26.64 MB/sec execute 243 sec latency 101.007 ms 2024-03-14T21:33:24.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 181772 26.58 MB/sec execute 244 sec latency 129.250 ms 2024-03-14T21:33:25.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 182029 26.51 MB/sec execute 245 sec latency 318.491 ms 2024-03-14T21:33:26.750 INFO:tasks.workunit.client.0.smithi163.stdout: 10 182473 26.48 MB/sec execute 246 sec latency 228.429 ms 2024-03-14T21:33:27.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 183188 26.50 MB/sec execute 247 sec latency 100.633 ms 2024-03-14T21:33:28.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 183877 26.58 MB/sec execute 248 sec latency 39.770 ms 2024-03-14T21:33:29.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 184487 26.67 MB/sec execute 249 sec latency 44.309 ms 2024-03-14T21:33:30.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 184786 26.64 MB/sec execute 250 sec latency 71.404 ms 2024-03-14T21:33:31.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 184894 26.55 MB/sec execute 251 sec latency 431.596 ms 2024-03-14T21:33:32.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 185122 26.46 MB/sec execute 252 sec latency 143.910 ms 2024-03-14T21:33:33.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 185519 26.42 MB/sec execute 253 sec latency 98.353 ms 2024-03-14T21:33:34.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 186127 26.41 MB/sec execute 254 sec latency 99.586 ms 2024-03-14T21:33:35.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 186836 26.45 MB/sec execute 255 sec latency 43.893 ms 2024-03-14T21:33:36.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 187372 26.48 MB/sec execute 256 sec latency 226.523 ms 2024-03-14T21:33:37.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 187897 26.57 MB/sec execute 257 sec latency 84.321 ms 2024-03-14T21:33:38.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 188306 26.55 MB/sec execute 258 sec latency 47.820 ms 2024-03-14T21:33:39.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 188514 26.47 MB/sec execute 259 sec latency 78.215 ms 2024-03-14T21:33:40.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 188829 26.40 MB/sec execute 260 sec latency 103.589 ms 2024-03-14T21:33:41.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 189155 26.36 MB/sec execute 261 sec latency 253.733 ms 2024-03-14T21:33:42.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 189747 26.34 MB/sec execute 262 sec latency 89.093 ms 2024-03-14T21:33:43.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 190653 26.44 MB/sec execute 263 sec latency 32.991 ms 2024-03-14T21:33:44.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 191358 26.56 MB/sec execute 264 sec latency 42.586 ms 2024-03-14T21:33:45.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 191807 26.57 MB/sec execute 265 sec latency 67.860 ms 2024-03-14T21:33:46.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 192022 26.49 MB/sec execute 266 sec latency 90.786 ms 2024-03-14T21:33:47.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 192160 26.40 MB/sec execute 267 sec latency 368.561 ms 2024-03-14T21:33:48.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 192466 26.34 MB/sec execute 268 sec latency 120.895 ms 2024-03-14T21:33:49.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 193053 26.36 MB/sec execute 269 sec latency 71.929 ms 2024-03-14T21:33:50.754 INFO:tasks.workunit.client.0.smithi163.stdout: 10 193919 26.38 MB/sec execute 270 sec latency 60.908 ms 2024-03-14T21:33:51.754 INFO:tasks.workunit.client.0.smithi163.stdout: 10 194674 26.49 MB/sec execute 271 sec latency 39.551 ms 2024-03-14T21:33:52.754 INFO:tasks.workunit.client.0.smithi163.stdout: 10 195040 26.51 MB/sec execute 272 sec latency 409.491 ms 2024-03-14T21:33:53.754 INFO:tasks.workunit.client.0.smithi163.stdout: 10 195363 26.49 MB/sec execute 273 sec latency 76.742 ms 2024-03-14T21:33:54.754 INFO:tasks.workunit.client.0.smithi163.stdout: 10 195630 26.42 MB/sec execute 274 sec latency 76.195 ms 2024-03-14T21:33:55.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 195948 26.36 MB/sec execute 275 sec latency 118.376 ms 2024-03-14T21:33:56.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 196455 26.35 MB/sec execute 276 sec latency 71.877 ms 2024-03-14T21:33:57.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 197047 26.34 MB/sec execute 277 sec latency 220.061 ms 2024-03-14T21:33:58.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 197727 26.37 MB/sec execute 278 sec latency 238.681 ms 2024-03-14T21:33:59.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 198433 26.48 MB/sec execute 279 sec latency 38.006 ms 2024-03-14T21:34:00.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 198887 26.50 MB/sec execute 280 sec latency 67.403 ms 2024-03-14T21:34:01.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 199135 26.44 MB/sec execute 281 sec latency 70.323 ms 2024-03-14T21:34:02.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 199403 26.37 MB/sec execute 282 sec latency 153.857 ms 2024-03-14T21:34:03.755 INFO:tasks.workunit.client.0.smithi163.stdout: 10 199744 26.32 MB/sec execute 283 sec latency 237.519 ms 2024-03-14T21:34:04.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 200322 26.32 MB/sec execute 284 sec latency 64.943 ms 2024-03-14T21:34:05.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 201040 26.35 MB/sec execute 285 sec latency 38.939 ms 2024-03-14T21:34:06.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 201657 26.41 MB/sec execute 286 sec latency 121.122 ms 2024-03-14T21:34:07.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 202228 26.49 MB/sec execute 287 sec latency 38.165 ms 2024-03-14T21:34:08.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 202413 26.42 MB/sec execute 288 sec latency 320.955 ms 2024-03-14T21:34:09.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 202635 26.36 MB/sec execute 289 sec latency 87.152 ms 2024-03-14T21:34:10.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 202905 26.30 MB/sec execute 290 sec latency 102.712 ms 2024-03-14T21:34:11.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 203374 26.27 MB/sec execute 291 sec latency 75.868 ms 2024-03-14T21:34:12.756 INFO:tasks.workunit.client.0.smithi163.stdout: 10 203952 26.27 MB/sec execute 292 sec latency 78.971 ms 2024-03-14T21:34:13.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 204489 26.25 MB/sec execute 293 sec latency 218.744 ms 2024-03-14T21:34:14.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 205103 26.32 MB/sec execute 294 sec latency 71.605 ms 2024-03-14T21:34:15.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 205782 26.41 MB/sec execute 295 sec latency 38.141 ms 2024-03-14T21:34:16.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 206115 26.38 MB/sec execute 296 sec latency 95.669 ms 2024-03-14T21:34:17.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 206380 26.31 MB/sec execute 297 sec latency 168.599 ms 2024-03-14T21:34:18.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 206644 26.26 MB/sec execute 298 sec latency 166.426 ms 2024-03-14T21:34:19.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 207101 26.24 MB/sec execute 299 sec latency 106.523 ms 2024-03-14T21:34:20.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 207812 26.25 MB/sec execute 300 sec latency 58.018 ms 2024-03-14T21:34:21.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 208417 26.29 MB/sec execute 301 sec latency 127.176 ms 2024-03-14T21:34:22.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 209040 26.35 MB/sec execute 302 sec latency 58.596 ms 2024-03-14T21:34:23.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 209470 26.38 MB/sec execute 303 sec latency 47.783 ms 2024-03-14T21:34:24.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 209628 26.31 MB/sec execute 304 sec latency 391.383 ms 2024-03-14T21:34:25.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 209841 26.24 MB/sec execute 305 sec latency 147.366 ms 2024-03-14T21:34:26.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 210169 26.20 MB/sec execute 306 sec latency 110.233 ms 2024-03-14T21:34:27.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 210731 26.19 MB/sec execute 307 sec latency 66.662 ms 2024-03-14T21:34:28.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 211511 26.21 MB/sec execute 308 sec latency 68.354 ms 2024-03-14T21:34:29.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 211909 26.22 MB/sec execute 309 sec latency 277.975 ms 2024-03-14T21:34:30.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 212446 26.26 MB/sec execute 310 sec latency 87.375 ms 2024-03-14T21:34:31.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 212929 26.28 MB/sec execute 311 sec latency 74.252 ms 2024-03-14T21:34:32.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 213224 26.25 MB/sec execute 312 sec latency 77.886 ms 2024-03-14T21:34:33.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 213539 26.20 MB/sec execute 313 sec latency 98.561 ms 2024-03-14T21:34:34.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 213936 26.17 MB/sec execute 314 sec latency 189.931 ms 2024-03-14T21:34:35.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 214576 26.16 MB/sec execute 315 sec latency 95.789 ms 2024-03-14T21:34:36.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 215233 26.21 MB/sec execute 316 sec latency 49.289 ms 2024-03-14T21:34:37.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 215976 26.28 MB/sec execute 317 sec latency 33.888 ms 2024-03-14T21:34:38.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 216418 26.28 MB/sec execute 318 sec latency 82.732 ms 2024-03-14T21:34:39.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 216647 26.24 MB/sec execute 319 sec latency 243.783 ms 2024-03-14T21:34:40.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 216948 26.20 MB/sec execute 320 sec latency 126.783 ms 2024-03-14T21:34:41.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 217405 26.19 MB/sec execute 321 sec latency 74.803 ms 2024-03-14T21:34:42.760 INFO:tasks.workunit.client.0.smithi163.stdout: 10 218056 26.18 MB/sec execute 322 sec latency 37.214 ms 2024-03-14T21:34:43.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 218663 26.19 MB/sec execute 323 sec latency 79.517 ms 2024-03-14T21:34:44.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 219377 26.28 MB/sec execute 324 sec latency 50.582 ms 2024-03-14T21:34:45.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 219760 26.26 MB/sec execute 325 sec latency 244.184 ms 2024-03-14T21:34:46.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 220044 26.24 MB/sec execute 326 sec latency 81.850 ms 2024-03-14T21:34:47.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 220375 26.20 MB/sec execute 327 sec latency 150.888 ms 2024-03-14T21:34:48.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 220788 26.18 MB/sec execute 328 sec latency 112.616 ms 2024-03-14T21:34:49.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 221472 26.20 MB/sec execute 329 sec latency 51.845 ms 2024-03-14T21:34:50.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 222009 26.19 MB/sec execute 330 sec latency 83.197 ms 2024-03-14T21:34:51.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 222359 26.16 MB/sec execute 331 sec latency 167.979 ms 2024-03-14T21:34:52.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 223118 26.25 MB/sec execute 332 sec latency 40.359 ms 2024-03-14T21:34:53.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 223504 26.24 MB/sec execute 333 sec latency 70.240 ms 2024-03-14T21:34:54.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 223815 26.20 MB/sec execute 334 sec latency 139.896 ms 2024-03-14T21:34:55.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 224247 26.19 MB/sec execute 335 sec latency 89.908 ms 2024-03-14T21:34:56.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 224520 26.16 MB/sec execute 336 sec latency 200.099 ms 2024-03-14T21:34:57.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 225146 26.15 MB/sec execute 337 sec latency 48.882 ms 2024-03-14T21:34:58.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 225691 26.16 MB/sec execute 338 sec latency 91.542 ms 2024-03-14T21:34:59.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 226387 26.23 MB/sec execute 339 sec latency 41.816 ms 2024-03-14T21:35:00.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 226927 26.24 MB/sec execute 340 sec latency 68.281 ms 2024-03-14T21:35:01.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 227138 26.19 MB/sec execute 341 sec latency 357.428 ms 2024-03-14T21:35:02.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 227364 26.15 MB/sec execute 342 sec latency 448.749 ms 2024-03-14T21:35:03.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 227787 26.14 MB/sec execute 343 sec latency 108.339 ms 2024-03-14T21:35:04.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 228338 26.13 MB/sec execute 344 sec latency 53.080 ms 2024-03-14T21:35:05.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 228915 26.13 MB/sec execute 345 sec latency 37.473 ms 2024-03-14T21:35:06.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 229408 26.14 MB/sec execute 346 sec latency 79.629 ms 2024-03-14T21:35:07.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 229739 26.14 MB/sec execute 347 sec latency 363.955 ms 2024-03-14T21:35:08.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 230183 26.15 MB/sec execute 348 sec latency 81.299 ms 2024-03-14T21:35:09.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 230585 26.12 MB/sec execute 349 sec latency 91.728 ms 2024-03-14T21:35:10.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 230952 26.10 MB/sec execute 350 sec latency 61.247 ms 2024-03-14T21:35:11.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 231435 26.10 MB/sec execute 351 sec latency 91.646 ms 2024-03-14T21:35:12.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 232007 26.10 MB/sec execute 352 sec latency 69.473 ms 2024-03-14T21:35:13.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 232302 26.06 MB/sec execute 353 sec latency 229.545 ms 2024-03-14T21:35:14.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 232797 26.08 MB/sec execute 354 sec latency 93.077 ms 2024-03-14T21:35:15.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 233430 26.11 MB/sec execute 355 sec latency 53.105 ms 2024-03-14T21:35:16.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 233850 26.11 MB/sec execute 356 sec latency 83.060 ms 2024-03-14T21:35:17.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 234241 26.08 MB/sec execute 357 sec latency 91.772 ms 2024-03-14T21:35:18.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 234507 26.06 MB/sec execute 358 sec latency 319.477 ms 2024-03-14T21:35:19.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 234892 26.04 MB/sec execute 359 sec latency 230.336 ms 2024-03-14T21:35:20.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 235384 26.03 MB/sec execute 360 sec latency 89.031 ms 2024-03-14T21:35:21.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 235959 26.02 MB/sec execute 361 sec latency 62.458 ms 2024-03-14T21:35:22.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 236570 26.07 MB/sec execute 362 sec latency 75.077 ms 2024-03-14T21:35:23.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 237205 26.11 MB/sec execute 363 sec latency 39.544 ms 2024-03-14T21:35:24.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 237482 26.08 MB/sec execute 364 sec latency 329.745 ms 2024-03-14T21:35:25.766 INFO:tasks.workunit.client.0.smithi163.stdout: 10 237801 26.04 MB/sec execute 365 sec latency 98.167 ms 2024-03-14T21:35:26.766 INFO:tasks.workunit.client.0.smithi163.stdout: 10 238178 26.02 MB/sec execute 366 sec latency 85.718 ms 2024-03-14T21:35:27.766 INFO:tasks.workunit.client.0.smithi163.stdout: 10 238721 26.03 MB/sec execute 367 sec latency 69.531 ms 2024-03-14T21:35:28.766 INFO:tasks.workunit.client.0.smithi163.stdout: 10 239305 26.03 MB/sec execute 368 sec latency 104.181 ms 2024-03-14T21:35:29.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 239620 26.00 MB/sec execute 369 sec latency 260.120 ms 2024-03-14T21:35:30.766 INFO:tasks.workunit.client.0.smithi163.stdout: 10 240077 26.01 MB/sec execute 370 sec latency 91.286 ms 2024-03-14T21:35:31.766 INFO:tasks.workunit.client.0.smithi163.stdout: 10 240651 26.04 MB/sec execute 371 sec latency 74.692 ms 2024-03-14T21:35:32.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 241114 26.03 MB/sec execute 372 sec latency 91.767 ms 2024-03-14T21:35:33.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 241466 26.02 MB/sec execute 373 sec latency 126.839 ms 2024-03-14T21:35:34.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 241699 25.98 MB/sec execute 374 sec latency 252.835 ms 2024-03-14T21:35:35.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 242103 25.96 MB/sec execute 375 sec latency 105.980 ms 2024-03-14T21:35:36.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 242500 25.95 MB/sec execute 376 sec latency 122.948 ms 2024-03-14T21:35:37.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 243069 25.93 MB/sec execute 377 sec latency 99.832 ms 2024-03-14T21:35:38.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 243707 25.98 MB/sec execute 378 sec latency 46.960 ms 2024-03-14T21:35:39.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 244155 25.98 MB/sec execute 379 sec latency 333.586 ms 2024-03-14T21:35:40.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 244618 25.98 MB/sec execute 380 sec latency 70.609 ms 2024-03-14T21:35:41.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 245011 25.98 MB/sec execute 381 sec latency 96.231 ms 2024-03-14T21:35:42.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 245416 25.96 MB/sec execute 382 sec latency 80.531 ms 2024-03-14T21:35:43.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 245884 25.96 MB/sec execute 383 sec latency 98.182 ms 2024-03-14T21:35:44.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 246325 25.93 MB/sec execute 384 sec latency 225.517 ms 2024-03-14T21:35:45.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 246825 25.93 MB/sec execute 385 sec latency 334.406 ms 2024-03-14T21:35:46.768 INFO:tasks.workunit.client.0.smithi163.stdout: 10 247373 25.95 MB/sec execute 386 sec latency 111.347 ms 2024-03-14T21:35:47.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 248026 25.99 MB/sec execute 387 sec latency 69.084 ms 2024-03-14T21:35:48.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 248453 25.98 MB/sec execute 388 sec latency 84.505 ms 2024-03-14T21:35:49.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 248801 25.96 MB/sec execute 389 sec latency 160.786 ms 2024-03-14T21:35:50.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 249198 25.94 MB/sec execute 390 sec latency 355.969 ms 2024-03-14T21:35:51.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 249741 25.95 MB/sec execute 391 sec latency 66.125 ms 2024-03-14T21:35:52.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 250361 25.95 MB/sec execute 392 sec latency 72.916 ms 2024-03-14T21:35:53.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 251085 26.00 MB/sec execute 393 sec latency 57.419 ms 2024-03-14T21:35:54.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 251659 26.03 MB/sec execute 394 sec latency 104.882 ms 2024-03-14T21:35:55.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 251938 26.00 MB/sec execute 395 sec latency 397.450 ms 2024-03-14T21:35:56.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 252259 25.98 MB/sec execute 396 sec latency 79.161 ms 2024-03-14T21:35:57.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 252711 25.96 MB/sec execute 397 sec latency 115.991 ms 2024-03-14T21:35:58.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 253260 25.97 MB/sec execute 398 sec latency 43.514 ms 2024-03-14T21:35:59.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 253905 25.98 MB/sec execute 399 sec latency 76.215 ms 2024-03-14T21:36:00.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 254291 25.97 MB/sec execute 400 sec latency 268.578 ms 2024-03-14T21:36:01.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 254774 25.99 MB/sec execute 401 sec latency 95.538 ms 2024-03-14T21:36:02.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 255229 25.99 MB/sec execute 402 sec latency 69.764 ms 2024-03-14T21:36:03.771 INFO:tasks.workunit.client.0.smithi163.stdout: 10 255612 25.97 MB/sec execute 403 sec latency 106.999 ms 2024-03-14T21:36:04.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 255960 25.95 MB/sec execute 404 sec latency 89.067 ms 2024-03-14T21:36:05.771 INFO:tasks.workunit.client.0.smithi163.stdout: 10 256346 25.94 MB/sec execute 405 sec latency 226.083 ms 2024-03-14T21:36:06.771 INFO:tasks.workunit.client.0.smithi163.stdout: 10 256819 25.93 MB/sec execute 406 sec latency 77.967 ms 2024-03-14T21:36:07.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 257510 25.95 MB/sec execute 407 sec latency 62.104 ms 2024-03-14T21:36:08.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 258197 25.99 MB/sec execute 408 sec latency 38.449 ms 2024-03-14T21:36:09.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 258711 26.01 MB/sec execute 409 sec latency 58.458 ms 2024-03-14T21:36:10.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 259073 25.99 MB/sec execute 410 sec latency 231.142 ms 2024-03-14T21:36:11.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 259353 25.96 MB/sec execute 411 sec latency 456.527 ms 2024-03-14T21:36:12.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 259701 25.94 MB/sec execute 412 sec latency 100.321 ms 2024-03-14T21:36:13.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 260211 25.93 MB/sec execute 413 sec latency 91.058 ms 2024-03-14T21:36:14.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 260750 25.94 MB/sec execute 414 sec latency 42.750 ms 2024-03-14T21:36:15.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 261274 25.94 MB/sec execute 415 sec latency 79.780 ms 2024-03-14T21:36:16.772 INFO:tasks.workunit.client.0.smithi163.stdout: 10 261593 25.92 MB/sec execute 416 sec latency 285.018 ms 2024-03-14T21:36:17.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 262037 25.94 MB/sec execute 417 sec latency 82.146 ms 2024-03-14T21:36:18.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 262566 25.93 MB/sec execute 418 sec latency 43.953 ms 2024-03-14T21:36:19.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 262981 25.93 MB/sec execute 419 sec latency 125.128 ms 2024-03-14T21:36:20.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 263499 25.92 MB/sec execute 420 sec latency 75.280 ms 2024-03-14T21:36:21.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 263863 25.90 MB/sec execute 421 sec latency 326.516 ms 2024-03-14T21:36:22.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 264480 25.92 MB/sec execute 422 sec latency 89.238 ms 2024-03-14T21:36:23.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 265074 25.93 MB/sec execute 423 sec latency 42.635 ms 2024-03-14T21:36:24.773 INFO:tasks.workunit.client.0.smithi163.stdout: 10 265603 25.97 MB/sec execute 424 sec latency 52.082 ms 2024-03-14T21:36:25.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 266101 25.96 MB/sec execute 425 sec latency 84.441 ms 2024-03-14T21:36:26.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 266314 25.92 MB/sec execute 426 sec latency 214.563 ms 2024-03-14T21:36:27.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 266591 25.90 MB/sec execute 427 sec latency 361.819 ms 2024-03-14T21:36:28.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 267049 25.88 MB/sec execute 428 sec latency 112.955 ms 2024-03-14T21:36:29.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 267518 25.88 MB/sec execute 429 sec latency 73.652 ms 2024-03-14T21:36:30.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 268151 25.89 MB/sec execute 430 sec latency 58.108 ms 2024-03-14T21:36:31.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 268492 25.87 MB/sec execute 431 sec latency 250.447 ms 2024-03-14T21:36:32.774 INFO:tasks.workunit.client.0.smithi163.stdout: 10 268837 25.87 MB/sec execute 432 sec latency 344.141 ms 2024-03-14T21:36:33.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 269319 25.89 MB/sec execute 433 sec latency 72.355 ms 2024-03-14T21:36:34.775 INFO:tasks.workunit.client.0.smithi163.stdout: 10 269745 25.87 MB/sec execute 434 sec latency 94.250 ms 2024-03-14T21:36:35.775 INFO:tasks.workunit.client.0.smithi163.stdout: 10 270187 25.87 MB/sec execute 435 sec latency 68.805 ms 2024-03-14T21:36:36.775 INFO:tasks.workunit.client.0.smithi163.stdout: 10 270626 25.85 MB/sec execute 436 sec latency 117.857 ms 2024-03-14T21:36:37.775 INFO:tasks.workunit.client.0.smithi163.stdout: 10 271071 25.85 MB/sec execute 437 sec latency 213.938 ms 2024-03-14T21:36:38.776 INFO:tasks.workunit.client.0.smithi163.stdout: 10 271769 25.87 MB/sec execute 438 sec latency 74.145 ms 2024-03-14T21:36:39.776 INFO:tasks.workunit.client.0.smithi163.stdout: 10 272311 25.88 MB/sec execute 439 sec latency 83.858 ms 2024-03-14T21:36:40.776 INFO:tasks.workunit.client.0.smithi163.stdout: 10 272834 25.90 MB/sec execute 440 sec latency 38.752 ms 2024-03-14T21:36:41.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 273218 25.89 MB/sec execute 441 sec latency 128.394 ms 2024-03-14T21:36:42.776 INFO:tasks.workunit.client.0.smithi163.stdout: 10 273448 25.86 MB/sec execute 442 sec latency 342.981 ms 2024-03-14T21:36:43.776 INFO:tasks.workunit.client.0.smithi163.stdout: 10 273840 25.85 MB/sec execute 443 sec latency 81.458 ms 2024-03-14T21:36:44.776 INFO:tasks.workunit.client.0.smithi163.stdout: 10 274284 25.83 MB/sec execute 444 sec latency 115.236 ms 2024-03-14T21:36:45.776 INFO:tasks.workunit.client.0.smithi163.stdout: 10 274912 25.85 MB/sec execute 445 sec latency 58.481 ms 2024-03-14T21:36:46.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 275560 25.86 MB/sec execute 446 sec latency 66.618 ms 2024-03-14T21:36:47.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 275998 25.87 MB/sec execute 447 sec latency 265.308 ms 2024-03-14T21:36:48.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 276440 25.88 MB/sec execute 448 sec latency 228.768 ms 2024-03-14T21:36:49.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 276853 25.86 MB/sec execute 449 sec latency 91.523 ms 2024-03-14T21:36:50.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 277286 25.86 MB/sec execute 450 sec latency 42.523 ms 2024-03-14T21:36:51.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 277762 25.84 MB/sec execute 451 sec latency 110.031 ms 2024-03-14T21:36:52.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 278429 25.87 MB/sec execute 452 sec latency 45.447 ms 2024-03-14T21:36:53.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 278939 25.86 MB/sec execute 453 sec latency 275.808 ms 2024-03-14T21:36:54.777 INFO:tasks.workunit.client.0.smithi163.stdout: 10 279378 25.86 MB/sec execute 454 sec latency 81.351 ms 2024-03-14T21:36:55.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 279991 25.90 MB/sec execute 455 sec latency 42.394 ms 2024-03-14T21:36:56.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 280395 25.88 MB/sec execute 456 sec latency 101.441 ms 2024-03-14T21:36:57.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 280792 25.88 MB/sec execute 457 sec latency 91.598 ms 2024-03-14T21:36:58.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 281176 25.85 MB/sec execute 458 sec latency 182.996 ms 2024-03-14T21:36:59.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 281652 25.84 MB/sec execute 459 sec latency 90.301 ms 2024-03-14T21:37:00.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 282428 25.88 MB/sec execute 460 sec latency 32.298 ms 2024-03-14T21:37:01.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 282985 25.90 MB/sec execute 461 sec latency 55.272 ms 2024-03-14T21:37:02.778 INFO:tasks.workunit.client.0.smithi163.stdout: 10 283588 25.93 MB/sec execute 462 sec latency 57.224 ms 2024-03-14T21:37:03.779 INFO:tasks.workunit.client.0.smithi163.stdout: 10 283869 25.90 MB/sec execute 463 sec latency 285.831 ms 2024-03-14T21:37:04.779 INFO:tasks.workunit.client.0.smithi163.stdout: 10 284214 25.88 MB/sec execute 464 sec latency 97.403 ms 2024-03-14T21:37:05.779 INFO:tasks.workunit.client.0.smithi163.stdout: 10 284811 25.88 MB/sec execute 465 sec latency 60.388 ms 2024-03-14T21:37:06.779 INFO:tasks.workunit.client.0.smithi163.stdout: 10 285376 25.90 MB/sec execute 466 sec latency 52.336 ms 2024-03-14T21:37:07.779 INFO:tasks.workunit.client.0.smithi163.stdout: 10 286136 25.92 MB/sec execute 467 sec latency 41.570 ms 2024-03-14T21:37:08.779 INFO:tasks.workunit.client.0.smithi163.stdout: 10 286738 25.95 MB/sec execute 468 sec latency 53.691 ms 2024-03-14T21:37:09.779 INFO:tasks.workunit.client.0.smithi163.stdout: 10 287117 25.95 MB/sec execute 469 sec latency 271.983 ms 2024-03-14T21:37:10.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 287496 25.93 MB/sec execute 470 sec latency 69.674 ms 2024-03-14T21:37:11.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 287944 25.93 MB/sec execute 471 sec latency 63.591 ms 2024-03-14T21:37:12.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 288411 25.90 MB/sec execute 472 sec latency 71.333 ms 2024-03-14T21:37:13.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 289042 25.93 MB/sec execute 473 sec latency 48.369 ms 2024-03-14T21:37:14.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 289553 25.92 MB/sec execute 474 sec latency 226.231 ms 2024-03-14T21:37:15.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 290140 25.95 MB/sec execute 475 sec latency 60.745 ms 2024-03-14T21:37:16.191 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:37:16.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:37:16.197 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:37:16.352 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-03-14T21:37:16.373+0000 7f5aad84b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46412) UID: 0 2024-03-14T21:37:16.352 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T21:37:16.373+0000 7f137ed27640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46412) UID: 0 2024-03-14T21:37:16.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 290738 25.97 MB/sec execute 476 sec latency 53.886 ms 2024-03-14T21:37:17.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 291083 25.96 MB/sec execute 477 sec latency 137.630 ms 2024-03-14T21:37:18.780 INFO:tasks.workunit.client.0.smithi163.stdout: 10 291450 25.94 MB/sec execute 478 sec latency 99.311 ms 2024-03-14T21:37:19.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 291792 25.92 MB/sec execute 479 sec latency 120.621 ms 2024-03-14T21:37:20.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 292001 25.88 MB/sec execute 480 sec latency 384.468 ms 2024-03-14T21:37:21.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 292465 25.88 MB/sec execute 481 sec latency 90.367 ms 2024-03-14T21:37:22.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 293018 25.87 MB/sec execute 482 sec latency 93.073 ms 2024-03-14T21:37:23.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 293579 25.89 MB/sec execute 483 sec latency 60.653 ms 2024-03-14T21:37:24.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 294157 25.91 MB/sec execute 484 sec latency 51.210 ms 2024-03-14T21:37:25.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 294466 25.91 MB/sec execute 485 sec latency 391.856 ms 2024-03-14T21:37:26.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 294734 25.88 MB/sec execute 486 sec latency 155.757 ms 2024-03-14T21:37:27.781 INFO:tasks.workunit.client.0.smithi163.stdout: 10 295171 25.87 MB/sec execute 487 sec latency 116.637 ms 2024-03-14T21:37:28.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 295664 25.86 MB/sec execute 488 sec latency 88.059 ms 2024-03-14T21:37:29.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 296308 25.88 MB/sec execute 489 sec latency 47.119 ms 2024-03-14T21:37:30.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 296704 25.86 MB/sec execute 490 sec latency 338.095 ms 2024-03-14T21:37:31.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 297155 25.87 MB/sec execute 491 sec latency 351.218 ms 2024-03-14T21:37:32.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 297685 25.89 MB/sec execute 492 sec latency 49.466 ms 2024-03-14T21:37:33.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 298095 25.88 MB/sec execute 493 sec latency 86.940 ms 2024-03-14T21:37:34.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 298442 25.86 MB/sec execute 494 sec latency 87.325 ms 2024-03-14T21:37:35.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 298790 25.85 MB/sec execute 495 sec latency 198.435 ms 2024-03-14T21:37:36.782 INFO:tasks.workunit.client.0.smithi163.stdout: 10 299146 25.82 MB/sec execute 496 sec latency 250.459 ms 2024-03-14T21:37:37.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 299704 25.83 MB/sec execute 497 sec latency 108.698 ms 2024-03-14T21:37:38.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 300334 25.84 MB/sec execute 498 sec latency 84.063 ms 2024-03-14T21:37:39.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 300966 25.88 MB/sec execute 499 sec latency 41.044 ms 2024-03-14T21:37:40.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 301419 25.87 MB/sec execute 500 sec latency 132.704 ms 2024-03-14T21:37:41.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 301656 25.85 MB/sec execute 501 sec latency 359.847 ms 2024-03-14T21:37:42.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 302006 25.83 MB/sec execute 502 sec latency 83.470 ms 2024-03-14T21:37:43.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 302439 25.83 MB/sec execute 503 sec latency 122.482 ms 2024-03-14T21:37:44.783 INFO:tasks.workunit.client.0.smithi163.stdout: 10 302915 25.81 MB/sec execute 504 sec latency 75.034 ms 2024-03-14T21:37:45.784 INFO:tasks.workunit.client.0.smithi163.stdout: 10 303434 25.82 MB/sec execute 505 sec latency 42.271 ms 2024-03-14T21:37:46.784 INFO:tasks.workunit.client.0.smithi163.stdout: 10 303835 25.80 MB/sec execute 506 sec latency 336.518 ms 2024-03-14T21:37:47.784 INFO:tasks.workunit.client.0.smithi163.stdout: 10 304366 25.83 MB/sec execute 507 sec latency 346.989 ms 2024-03-14T21:37:48.784 INFO:tasks.workunit.client.0.smithi163.stdout: 10 304886 25.84 MB/sec execute 508 sec latency 78.481 ms 2024-03-14T21:37:49.784 INFO:tasks.workunit.client.0.smithi163.stdout: 10 305291 25.83 MB/sec execute 509 sec latency 87.597 ms 2024-03-14T21:37:50.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 305712 25.82 MB/sec execute 510 sec latency 57.281 ms 2024-03-14T21:37:51.784 INFO:tasks.workunit.client.0.smithi163.stdout: 10 306126 25.81 MB/sec execute 511 sec latency 132.449 ms 2024-03-14T21:37:52.784 INFO:tasks.workunit.client.0.smithi163.stdout: 10 306544 25.79 MB/sec execute 512 sec latency 193.343 ms 2024-03-14T21:37:53.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 307148 25.81 MB/sec execute 513 sec latency 73.153 ms 2024-03-14T21:37:54.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 307772 25.82 MB/sec execute 514 sec latency 87.496 ms 2024-03-14T21:37:55.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 308256 25.83 MB/sec execute 515 sec latency 84.339 ms 2024-03-14T21:37:56.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 308703 25.83 MB/sec execute 516 sec latency 122.730 ms 2024-03-14T21:37:57.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 308981 25.81 MB/sec execute 517 sec latency 217.265 ms 2024-03-14T21:37:58.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 309413 25.81 MB/sec execute 518 sec latency 103.216 ms 2024-03-14T21:37:59.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 309973 25.80 MB/sec execute 519 sec latency 100.267 ms 2024-03-14T21:38:00.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 310465 25.81 MB/sec execute 520 sec latency 93.679 ms 2024-03-14T21:38:01.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 311080 25.82 MB/sec execute 521 sec latency 74.214 ms 2024-03-14T21:38:02.785 INFO:tasks.workunit.client.0.smithi163.stdout: 10 311346 25.80 MB/sec execute 522 sec latency 252.751 ms 2024-03-14T21:38:03.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 311891 25.81 MB/sec execute 523 sec latency 66.259 ms 2024-03-14T21:38:04.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 312354 25.81 MB/sec execute 524 sec latency 106.690 ms 2024-03-14T21:38:05.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 312813 25.81 MB/sec execute 525 sec latency 125.376 ms 2024-03-14T21:38:06.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 313215 25.79 MB/sec execute 526 sec latency 96.315 ms 2024-03-14T21:38:07.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 313555 25.77 MB/sec execute 527 sec latency 248.397 ms 2024-03-14T21:38:08.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 313962 25.78 MB/sec execute 528 sec latency 101.461 ms 2024-03-14T21:38:09.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 314491 25.77 MB/sec execute 529 sec latency 111.965 ms 2024-03-14T21:38:10.786 INFO:tasks.workunit.client.0.smithi163.stdout: 10 314955 25.77 MB/sec execute 530 sec latency 77.167 ms 2024-03-14T21:38:11.787 INFO:tasks.workunit.client.0.smithi163.stdout: 10 315398 25.78 MB/sec execute 531 sec latency 113.324 ms 2024-03-14T21:38:12.787 INFO:tasks.workunit.client.0.smithi163.stdout: 10 315766 25.77 MB/sec execute 532 sec latency 313.007 ms 2024-03-14T21:38:13.787 INFO:tasks.workunit.client.0.smithi163.stdout: 10 316157 25.76 MB/sec execute 533 sec latency 331.263 ms 2024-03-14T21:38:14.787 INFO:tasks.workunit.client.0.smithi163.stdout: 10 316608 25.76 MB/sec execute 534 sec latency 68.234 ms 2024-03-14T21:38:15.787 INFO:tasks.workunit.client.0.smithi163.stdout: 10 317105 25.75 MB/sec execute 535 sec latency 68.556 ms 2024-03-14T21:38:16.787 INFO:tasks.workunit.client.0.smithi163.stdout: 10 317597 25.76 MB/sec execute 536 sec latency 108.554 ms 2024-03-14T21:38:17.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 318228 25.77 MB/sec execute 537 sec latency 101.585 ms 2024-03-14T21:38:18.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 318558 25.75 MB/sec execute 538 sec latency 267.510 ms 2024-03-14T21:38:19.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 319018 25.75 MB/sec execute 539 sec latency 111.135 ms 2024-03-14T21:38:20.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 319573 25.77 MB/sec execute 540 sec latency 54.214 ms 2024-03-14T21:38:21.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 320042 25.77 MB/sec execute 541 sec latency 116.085 ms 2024-03-14T21:38:22.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 320606 25.76 MB/sec execute 542 sec latency 73.741 ms 2024-03-14T21:38:23.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 320843 25.74 MB/sec execute 543 sec latency 281.267 ms 2024-03-14T21:38:24.788 INFO:tasks.workunit.client.0.smithi163.stdout: 10 321307 25.75 MB/sec execute 544 sec latency 91.924 ms 2024-03-14T21:38:25.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 321962 25.75 MB/sec execute 545 sec latency 63.196 ms 2024-03-14T21:38:26.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 322420 25.76 MB/sec execute 546 sec latency 85.386 ms 2024-03-14T21:38:27.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 322896 25.75 MB/sec execute 547 sec latency 89.326 ms 2024-03-14T21:38:28.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 323176 25.74 MB/sec execute 548 sec latency 233.738 ms 2024-03-14T21:38:29.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 323579 25.75 MB/sec execute 549 sec latency 97.714 ms 2024-03-14T21:38:30.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 324097 25.74 MB/sec execute 550 sec latency 88.265 ms 2024-03-14T21:38:31.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 324414 25.72 MB/sec execute 551 sec latency 120.271 ms 2024-03-14T21:38:32.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 324961 25.73 MB/sec execute 552 sec latency 56.456 ms 2024-03-14T21:38:33.789 INFO:tasks.workunit.client.0.smithi163.stdout: 10 325529 25.73 MB/sec execute 553 sec latency 91.674 ms 2024-03-14T21:38:34.790 INFO:tasks.workunit.client.0.smithi163.stdout: 10 325777 25.71 MB/sec execute 554 sec latency 324.322 ms 2024-03-14T21:38:35.790 INFO:tasks.workunit.client.0.smithi163.stdout: 10 326214 25.71 MB/sec execute 555 sec latency 87.416 ms 2024-03-14T21:38:36.790 INFO:tasks.workunit.client.0.smithi163.stdout: 10 326748 25.72 MB/sec execute 556 sec latency 64.068 ms 2024-03-14T21:38:37.790 INFO:tasks.workunit.client.0.smithi163.stdout: 10 327231 25.73 MB/sec execute 557 sec latency 98.150 ms 2024-03-14T21:38:38.790 INFO:tasks.workunit.client.0.smithi163.stdout: 10 327681 25.71 MB/sec execute 558 sec latency 74.847 ms 2024-03-14T21:38:39.790 INFO:tasks.workunit.client.0.smithi163.stdout: 10 327891 25.68 MB/sec execute 559 sec latency 287.877 ms 2024-03-14T21:38:40.790 INFO:tasks.workunit.client.0.smithi163.stdout: 10 328261 25.68 MB/sec execute 560 sec latency 392.945 ms 2024-03-14T21:38:41.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 328746 25.68 MB/sec execute 561 sec latency 161.690 ms 2024-03-14T21:38:42.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 329321 25.68 MB/sec execute 562 sec latency 58.695 ms 2024-03-14T21:38:43.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 329928 25.70 MB/sec execute 563 sec latency 59.515 ms 2024-03-14T21:38:44.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 330372 25.70 MB/sec execute 564 sec latency 142.956 ms 2024-03-14T21:38:45.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 330766 25.71 MB/sec execute 565 sec latency 283.217 ms 2024-03-14T21:38:46.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 331216 25.69 MB/sec execute 566 sec latency 69.239 ms 2024-03-14T21:38:47.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 331570 25.67 MB/sec execute 567 sec latency 126.033 ms 2024-03-14T21:38:48.791 INFO:tasks.workunit.client.0.smithi163.stdout: 10 332092 25.68 MB/sec execute 568 sec latency 61.749 ms 2024-03-14T21:38:49.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 332689 25.69 MB/sec execute 569 sec latency 92.340 ms 2024-03-14T21:38:50.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 333056 25.68 MB/sec execute 570 sec latency 230.579 ms 2024-03-14T21:38:51.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 333573 25.68 MB/sec execute 571 sec latency 87.840 ms 2024-03-14T21:38:52.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 334118 25.70 MB/sec execute 572 sec latency 84.309 ms 2024-03-14T21:38:53.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 334674 25.70 MB/sec execute 573 sec latency 49.625 ms 2024-03-14T21:38:54.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 335091 25.69 MB/sec execute 574 sec latency 118.297 ms 2024-03-14T21:38:55.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 335478 25.70 MB/sec execute 575 sec latency 223.355 ms 2024-03-14T21:38:56.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 336025 25.68 MB/sec execute 576 sec latency 86.064 ms 2024-03-14T21:38:57.792 INFO:tasks.workunit.client.0.smithi163.stdout: 10 336531 25.70 MB/sec execute 577 sec latency 39.077 ms 2024-03-14T21:38:58.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 337127 25.71 MB/sec execute 578 sec latency 45.824 ms 2024-03-14T21:38:59.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 337606 25.71 MB/sec execute 579 sec latency 80.871 ms 2024-03-14T21:39:00.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 337930 25.71 MB/sec execute 580 sec latency 187.162 ms 2024-03-14T21:39:01.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 338298 25.68 MB/sec execute 581 sec latency 127.954 ms 2024-03-14T21:39:02.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 338734 25.68 MB/sec execute 582 sec latency 89.939 ms 2024-03-14T21:39:03.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 339426 25.69 MB/sec execute 583 sec latency 39.618 ms 2024-03-14T21:39:04.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 339935 25.70 MB/sec execute 584 sec latency 75.721 ms 2024-03-14T21:39:05.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 340373 25.70 MB/sec execute 585 sec latency 208.520 ms 2024-03-14T21:39:06.793 INFO:tasks.workunit.client.0.smithi163.stdout: 10 340766 25.70 MB/sec execute 586 sec latency 78.212 ms 2024-03-14T21:39:07.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 341253 25.70 MB/sec execute 587 sec latency 72.798 ms 2024-03-14T21:39:08.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 341799 25.70 MB/sec execute 588 sec latency 38.477 ms 2024-03-14T21:39:09.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 342204 25.69 MB/sec execute 589 sec latency 80.222 ms 2024-03-14T21:39:10.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 342615 25.69 MB/sec execute 590 sec latency 201.260 ms 2024-03-14T21:39:11.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 343079 25.68 MB/sec execute 591 sec latency 167.583 ms 2024-03-14T21:39:12.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 343598 25.69 MB/sec execute 592 sec latency 67.093 ms 2024-03-14T21:39:13.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 344216 25.70 MB/sec execute 593 sec latency 45.573 ms 2024-03-14T21:39:14.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 344649 25.70 MB/sec execute 594 sec latency 79.562 ms 2024-03-14T21:39:15.794 INFO:tasks.workunit.client.0.smithi163.stdout: 10 344989 25.69 MB/sec execute 595 sec latency 297.557 ms 2024-03-14T21:39:16.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 345380 25.68 MB/sec execute 596 sec latency 227.784 ms 2024-03-14T21:39:17.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 345798 25.67 MB/sec execute 597 sec latency 100.580 ms 2024-03-14T21:39:18.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 346386 25.68 MB/sec execute 598 sec latency 40.822 ms 2024-03-14T21:39:19.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 346939 25.68 MB/sec execute 599 sec latency 99.579 ms 2024-03-14T21:39:20.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 600 sec 2024-03-14T21:39:21.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 601 sec 2024-03-14T21:39:22.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 602 sec 2024-03-14T21:39:23.795 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 603 sec 2024-03-14T21:39:24.795 INFO:tasks.workunit.client.0.smithi163.stdout: 9 cleanup 604 sec 2024-03-14T21:39:25.795 INFO:tasks.workunit.client.0.smithi163.stdout: 9 cleanup 605 sec 2024-03-14T21:39:26.796 INFO:tasks.workunit.client.0.smithi163.stdout: 8 cleanup 606 sec 2024-03-14T21:39:27.796 INFO:tasks.workunit.client.0.smithi163.stdout: 5 cleanup 607 sec 2024-03-14T21:39:28.796 INFO:tasks.workunit.client.0.smithi163.stdout: 1 cleanup 608 sec 2024-03-14T21:39:29.654 INFO:tasks.workunit.client.0.smithi163.stdout: 0 cleanup 609 sec 2024-03-14T21:39:29.654 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:39:29.654 INFO:tasks.workunit.client.0.smithi163.stdout: Operation Count AvgLat MaxLat 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: ---------------------------------------- 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: NTCreateX 493393 4.922 1986.765 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Close 362456 0.009 2.616 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Rename 20930 22.367 2124.668 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Unlink 99607 10.030 1733.503 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Qpathinfo 447653 3.320 1734.442 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Qfileinfo 78103 0.006 0.669 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Qfsinfo 82029 0.521 52.788 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Sfileinfo 40251 0.183 391.572 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: Find 172975 0.771 395.461 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: WriteX 243951 0.085 153.017 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: ReadX 774441 0.015 93.812 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: LockX 1610 7.887 204.193 2024-03-14T21:39:29.655 INFO:tasks.workunit.client.0.smithi163.stdout: UnlockX 1610 7.347 148.564 2024-03-14T21:39:29.656 INFO:tasks.workunit.client.0.smithi163.stdout: Flush 34579 10.396 273.302 2024-03-14T21:39:29.656 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:39:29.656 INFO:tasks.workunit.client.0.smithi163.stdout:Throughput 25.6846 MB/sec 10 clients 10 procs max_latency=456.527 ms 2024-03-14T21:39:29.656 INFO:tasks.workunit.client.0.smithi163.stderr:+ popd 2024-03-14T21:39:29.656 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-14T21:39:29.657 INFO:tasks.workunit.client.0.smithi163.stderr:+ clean_up 2024-03-14T21:39:29.657 INFO:tasks.workunit.client.0.smithi163.stderr:+ rm -rf xfprogs-dev-dir 2024-03-14T21:39:29.659 INFO:tasks.workunit.client.0.smithi163.stderr:+ rm -rf xfstest-dev-dir 2024-03-14T21:39:41.174 INFO:tasks.workunit.client.0.smithi163.stderr:+ rm -rf fscrypt_test_none_dbench 2024-03-14T21:39:41.965 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-14T21:39:41.965 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-14T21:39:42.475 INFO:tasks.workunit:Running workunits matching fs/fscrypt.sh on client.0... 2024-03-14T21:39:42.477 INFO:tasks.workunit:Running workunit fs/fscrypt.sh... 2024-03-14T21:39:42.477 DEBUG:teuthology.orchestra.run.smithi163:workunit test fs/fscrypt.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=234d532354068e06d01621fd032c3b663cead394 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh unlocked dbench 2024-03-14T21:39:42.558 INFO:tasks.workunit.client.0.smithi163.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh 2024-03-14T21:39:42.561 INFO:tasks.workunit.client.0.smithi163.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-03-14T21:39:42.561 INFO:tasks.workunit.client.0.smithi163.stderr:+ '[' 2 -ne 2 ']' 2024-03-14T21:39:42.561 INFO:tasks.workunit.client.0.smithi163.stderr:+ fscrypt=unlocked 2024-03-14T21:39:42.561 INFO:tasks.workunit.client.0.smithi163.stderr:+ testcase=dbench 2024-03-14T21:39:42.561 INFO:tasks.workunit.client.0.smithi163.stderr:+ testdir=fscrypt_test_unlocked_dbench 2024-03-14T21:39:42.561 INFO:tasks.workunit.client.0.smithi163.stderr:+ mkdir fscrypt_test_unlocked_dbench 2024-03-14T21:39:42.567 INFO:tasks.workunit.client.0.smithi163.stderr:+ XFSPROGS_DIR=xfprogs-dev-dir 2024-03-14T21:39:42.567 INFO:tasks.workunit.client.0.smithi163.stderr:+ XFSTESTS_DIR=xfstest-dev-dir 2024-03-14T21:39:42.567 INFO:tasks.workunit.client.0.smithi163.stderr:++ type -P xfs_io 2024-03-14T21:39:42.568 INFO:tasks.workunit.client.0.smithi163.stderr:+ export XFS_IO_PROG=/usr/sbin/xfs_io 2024-03-14T21:39:42.568 INFO:tasks.workunit.client.0.smithi163.stderr:+ XFS_IO_PROG=/usr/sbin/xfs_io 2024-03-14T21:39:42.568 INFO:tasks.workunit.client.0.smithi163.stderr:+ install_xfsprogs 2024-03-14T21:39:42.568 INFO:tasks.workunit.client.0.smithi163.stderr:+ local install_xfsprogs=0 2024-03-14T21:39:42.569 INFO:tasks.workunit.client.0.smithi163.stderr:+ xfs_io -c 'help add_enckey' 2024-03-14T21:39:42.569 INFO:tasks.workunit.client.0.smithi163.stderr:+ grep -q 'not found' 2024-03-14T21:39:42.574 INFO:tasks.workunit.client.0.smithi163.stderr:+ '[' 0 -eq 1 ']' 2024-03-14T21:39:42.574 INFO:tasks.workunit.client.0.smithi163.stderr:+ setup_xfstests_env 2024-03-14T21:39:42.574 INFO:tasks.workunit.client.0.smithi163.stderr:+ git clone https://git.ceph.com/xfstests-dev.git xfstest-dev-dir --depth 1 2024-03-14T21:39:42.579 INFO:tasks.workunit.client.0.smithi163.stderr:Cloning into 'xfstest-dev-dir'... 2024-03-14T21:39:49.147 INFO:tasks.workunit.client.0.smithi163.stderr:Updating files: 35% (1500/4261) Updating files: 36% (1534/4261) Updating files: 37% (1577/4261) Updating files: 38% (1620/4261) Updating files: 39% (1662/4261) Updating files: 40% (1705/4261) Updating files: 41% (1748/4261) Updating files: 42% (1790/4261) Updating files: 43% (1833/4261) Updating files: 44% (1875/4261) Updating files: 45% (1918/4261) Updating files: 46% (1961/4261) Updating files: 47% (2003/4261) Updating files: 48% (2046/4261) Updating files: 49% (2088/4261) Updating files: 50% (2131/4261) Updating files: 50% (2141/4261) Updating files: 51% (2174/4261) Updating files: 52% (2216/4261) Updating files: 53% (2259/4261) Updating files: 54% (2301/4261) Updating files: 55% (2344/4261) Updating files: 56% (2387/4261) Updating files: 57% (2429/4261) Updating files: 58% (2472/4261) Updating files: 59% (2514/4261) Updating files: 60% (2557/4261) Updating files: 61% (2600/4261) Updating files: 61% (2605/4261) Updating files: 62% (2642/4261) Updating files: 63% (2685/4261) Updating files: 64% (2728/4261) Updating files: 65% (2770/4261) Updating files: 66% (2813/4261) Updating files: 67% (2855/4261) Updating files: 68% (2898/4261) Updating files: 69% (2941/4261) Updating files: 70% (2983/4261) Updating files: 71% (3026/4261) Updating files: 72% (3068/4261) Updating files: 73% (3111/4261) Updating files: 74% (3154/4261) Updating files: 75% (3196/4261) Updating files: 76% (3239/4261) Updating files: 77% (3281/4261) Updating files: 78% (3324/4261) Updating files: 79% (3367/4261) Updating files: 79% (3406/4261) Updating files: 80% (3409/4261) Updating files: 81% (3452/4261) Updating files: 82% (3495/4261) Updating files: 83% (3537/4261) Updating files: 84% (3580/4261) Updating files: 85% (3622/4261) Updating files: 86% (3665/4261) Updating files: 87% (3708/4261) Updating files: 88% (3750/4261) Updating files: 89% (3793/4261) Updating files: 90% (3835/4261) Updating files: 91% (3878/4261) Updating files: 92% (3921/4261) Updating files: 93% (3963/4261) Updating files: 94% (4006/4261) Updating files: 95% (4048/4261) Updating files: 96% (4091/4261) Updating files: 97% (4134/4261) Updating files: 98% (4176/4261) Updating files: 99% (4219/4261) Updating files: 99% (4249/4261) Updating files: 100% (4261/4261) Updating files: 100% (4261/4261), done. 2024-03-14T21:39:49.160 INFO:tasks.workunit.client.0.smithi163.stderr:+ pushd xfstest-dev-dir 2024-03-14T21:39:49.160 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp/xfstest-dev-dir ~/cephtest/mnt.0/client.0/tmp 2024-03-14T21:39:49.161 INFO:tasks.workunit.client.0.smithi163.stderr:+ . common/encrypt 2024-03-14T21:39:49.161 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY= 2024-03-14T21:39:49.161 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.162 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 1 2024-03-14T21:39:49.163 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x01' 2024-03-14T21:39:49.163 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.164 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 2 2024-03-14T21:39:49.164 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x02' 2024-03-14T21:39:49.164 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.166 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 3 2024-03-14T21:39:49.166 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x03' 2024-03-14T21:39:49.167 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.168 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 4 2024-03-14T21:39:49.169 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x04' 2024-03-14T21:39:49.169 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.170 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 5 2024-03-14T21:39:49.171 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x05' 2024-03-14T21:39:49.171 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.172 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 6 2024-03-14T21:39:49.173 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x06' 2024-03-14T21:39:49.173 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.174 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 7 2024-03-14T21:39:49.175 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x07' 2024-03-14T21:39:49.175 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.176 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 8 2024-03-14T21:39:49.176 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x08' 2024-03-14T21:39:49.176 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.177 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 9 2024-03-14T21:39:49.178 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x09' 2024-03-14T21:39:49.178 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.179 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 10 2024-03-14T21:39:49.180 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0a' 2024-03-14T21:39:49.180 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.181 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 11 2024-03-14T21:39:49.182 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0b' 2024-03-14T21:39:49.182 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.183 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 12 2024-03-14T21:39:49.184 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0c' 2024-03-14T21:39:49.184 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.185 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 13 2024-03-14T21:39:49.186 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0d' 2024-03-14T21:39:49.186 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.187 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 14 2024-03-14T21:39:49.187 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0e' 2024-03-14T21:39:49.188 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.189 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 15 2024-03-14T21:39:49.189 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x0f' 2024-03-14T21:39:49.190 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.192 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 16 2024-03-14T21:39:49.192 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x10' 2024-03-14T21:39:49.192 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.194 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 17 2024-03-14T21:39:49.195 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x11' 2024-03-14T21:39:49.195 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.197 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 18 2024-03-14T21:39:49.198 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x12' 2024-03-14T21:39:49.198 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.199 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 19 2024-03-14T21:39:49.200 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x13' 2024-03-14T21:39:49.200 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.201 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 20 2024-03-14T21:39:49.202 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x14' 2024-03-14T21:39:49.202 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.203 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 21 2024-03-14T21:39:49.204 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x15' 2024-03-14T21:39:49.204 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.205 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 22 2024-03-14T21:39:49.206 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x16' 2024-03-14T21:39:49.206 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.207 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 23 2024-03-14T21:39:49.208 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x17' 2024-03-14T21:39:49.208 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.210 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 24 2024-03-14T21:39:49.210 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x18' 2024-03-14T21:39:49.210 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.212 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 25 2024-03-14T21:39:49.212 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x19' 2024-03-14T21:39:49.213 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.214 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 26 2024-03-14T21:39:49.214 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1a' 2024-03-14T21:39:49.215 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.216 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 27 2024-03-14T21:39:49.217 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1b' 2024-03-14T21:39:49.217 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.218 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 28 2024-03-14T21:39:49.219 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1c' 2024-03-14T21:39:49.219 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.220 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 29 2024-03-14T21:39:49.221 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1d' 2024-03-14T21:39:49.221 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.222 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 30 2024-03-14T21:39:49.223 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1e' 2024-03-14T21:39:49.223 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.224 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 31 2024-03-14T21:39:49.225 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x1f' 2024-03-14T21:39:49.225 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.226 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 32 2024-03-14T21:39:49.227 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x20' 2024-03-14T21:39:49.227 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.229 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 33 2024-03-14T21:39:49.229 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x21' 2024-03-14T21:39:49.230 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.231 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 34 2024-03-14T21:39:49.232 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x22' 2024-03-14T21:39:49.232 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.233 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 35 2024-03-14T21:39:49.234 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x23' 2024-03-14T21:39:49.234 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.235 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 36 2024-03-14T21:39:49.236 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x24' 2024-03-14T21:39:49.236 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.237 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 37 2024-03-14T21:39:49.238 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x25' 2024-03-14T21:39:49.238 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.239 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 38 2024-03-14T21:39:49.240 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x26' 2024-03-14T21:39:49.240 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.242 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 39 2024-03-14T21:39:49.242 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x27' 2024-03-14T21:39:49.242 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.244 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 40 2024-03-14T21:39:49.244 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x28' 2024-03-14T21:39:49.245 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.246 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 41 2024-03-14T21:39:49.247 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x29' 2024-03-14T21:39:49.247 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.248 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 42 2024-03-14T21:39:49.249 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2a' 2024-03-14T21:39:49.249 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.250 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 43 2024-03-14T21:39:49.251 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2b' 2024-03-14T21:39:49.251 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.252 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 44 2024-03-14T21:39:49.253 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2c' 2024-03-14T21:39:49.253 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.254 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 45 2024-03-14T21:39:49.255 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2d' 2024-03-14T21:39:49.255 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.257 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 46 2024-03-14T21:39:49.257 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2e' 2024-03-14T21:39:49.257 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.259 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 47 2024-03-14T21:39:49.260 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x2f' 2024-03-14T21:39:49.260 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.261 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 48 2024-03-14T21:39:49.262 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x30' 2024-03-14T21:39:49.262 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.264 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 49 2024-03-14T21:39:49.264 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x31' 2024-03-14T21:39:49.264 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.266 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 50 2024-03-14T21:39:49.267 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x32' 2024-03-14T21:39:49.267 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.268 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 51 2024-03-14T21:39:49.269 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x33' 2024-03-14T21:39:49.269 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.271 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 52 2024-03-14T21:39:49.272 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x34' 2024-03-14T21:39:49.272 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.273 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 53 2024-03-14T21:39:49.274 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x35' 2024-03-14T21:39:49.274 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.275 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 54 2024-03-14T21:39:49.276 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x36' 2024-03-14T21:39:49.276 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.277 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 55 2024-03-14T21:39:49.278 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x37' 2024-03-14T21:39:49.278 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.279 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 56 2024-03-14T21:39:49.280 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x38' 2024-03-14T21:39:49.280 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.282 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 57 2024-03-14T21:39:49.282 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x39' 2024-03-14T21:39:49.282 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.284 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 58 2024-03-14T21:39:49.285 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3a' 2024-03-14T21:39:49.285 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.286 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 59 2024-03-14T21:39:49.287 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3b' 2024-03-14T21:39:49.287 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.288 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 60 2024-03-14T21:39:49.289 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3c' 2024-03-14T21:39:49.289 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.290 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 61 2024-03-14T21:39:49.291 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3d' 2024-03-14T21:39:49.291 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.292 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 62 2024-03-14T21:39:49.293 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3e' 2024-03-14T21:39:49.293 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.294 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 63 2024-03-14T21:39:49.295 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x3f' 2024-03-14T21:39:49.295 INFO:tasks.workunit.client.0.smithi163.stderr:++ for i in {1..64} 2024-03-14T21:39:49.296 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 64 2024-03-14T21:39:49.297 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_RAW_KEY+='\x40' 2024-03-14T21:39:49.297 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_KEY_DESCRIPTOR=0000111122223333 2024-03-14T21:39:49.297 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_KEY_IDENTIFIER=69b2f6edeee720cce0577937eb8a6751 2024-03-14T21:39:49.298 INFO:tasks.workunit.client.0.smithi163.stderr:++ TEST_KEYRING_ID= 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_256_XTS=1 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_256_CTS=4 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_128_CBC=5 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_128_CTS=6 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_ADIANTUM=9 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_MODE_AES_256_HCTR2=10 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_POLICY_FLAG_DIRECT_KEY=0x04 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64=0x08 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32=0x10 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_KEY_SPEC_TYPE_DESCRIPTOR=1 2024-03-14T21:39:49.301 INFO:tasks.workunit.client.0.smithi163.stderr:++ FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER=2 2024-03-14T21:39:49.302 INFO:tasks.workunit.client.0.smithi163.stderr:+ popd 2024-03-14T21:39:49.302 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-14T21:39:49.304 INFO:tasks.workunit.client.0.smithi163.stderr:++ _generate_raw_encryption_key 2024-03-14T21:39:49.304 INFO:tasks.workunit.client.0.smithi163.stderr:++ local raw= 2024-03-14T21:39:49.304 INFO:tasks.workunit.client.0.smithi163.stderr:++ local i 2024-03-14T21:39:49.304 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i = 0 )) 2024-03-14T21:39:49.304 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.305 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 125 2024-03-14T21:39:49.306 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d' 2024-03-14T21:39:49.306 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.306 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.307 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 231 2024-03-14T21:39:49.308 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7' 2024-03-14T21:39:49.308 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.308 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.309 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 24 2024-03-14T21:39:49.309 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18' 2024-03-14T21:39:49.310 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.310 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.311 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 215 2024-03-14T21:39:49.312 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7' 2024-03-14T21:39:49.312 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.312 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.313 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 180 2024-03-14T21:39:49.314 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4' 2024-03-14T21:39:49.314 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.314 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.315 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 248 2024-03-14T21:39:49.315 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8' 2024-03-14T21:39:49.315 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.315 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.316 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 146 2024-03-14T21:39:49.317 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92' 2024-03-14T21:39:49.317 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.317 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.319 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 20 2024-03-14T21:39:49.319 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14' 2024-03-14T21:39:49.319 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.320 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.320 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 99 2024-03-14T21:39:49.321 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63' 2024-03-14T21:39:49.321 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.321 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.322 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 242 2024-03-14T21:39:49.323 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2' 2024-03-14T21:39:49.323 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.323 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.324 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 109 2024-03-14T21:39:49.325 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d' 2024-03-14T21:39:49.325 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.325 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.326 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 247 2024-03-14T21:39:49.327 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7' 2024-03-14T21:39:49.327 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.327 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.328 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 56 2024-03-14T21:39:49.329 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38' 2024-03-14T21:39:49.329 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.329 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.330 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 35 2024-03-14T21:39:49.330 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23' 2024-03-14T21:39:49.330 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.330 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.331 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 171 2024-03-14T21:39:49.332 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab' 2024-03-14T21:39:49.332 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.332 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.333 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 209 2024-03-14T21:39:49.334 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1' 2024-03-14T21:39:49.334 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.334 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.335 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 188 2024-03-14T21:39:49.335 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc' 2024-03-14T21:39:49.335 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.335 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.336 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 215 2024-03-14T21:39:49.337 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7' 2024-03-14T21:39:49.337 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.337 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.338 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 169 2024-03-14T21:39:49.339 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9' 2024-03-14T21:39:49.339 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.339 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.340 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 73 2024-03-14T21:39:49.340 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49' 2024-03-14T21:39:49.340 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.341 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.342 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 115 2024-03-14T21:39:49.342 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73' 2024-03-14T21:39:49.342 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.342 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.343 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 54 2024-03-14T21:39:49.344 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36' 2024-03-14T21:39:49.344 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.344 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.345 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 24 2024-03-14T21:39:49.346 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18' 2024-03-14T21:39:49.346 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.346 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.347 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 7 2024-03-14T21:39:49.347 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07' 2024-03-14T21:39:49.348 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.348 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.349 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 66 2024-03-14T21:39:49.350 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42' 2024-03-14T21:39:49.350 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.350 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.351 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 177 2024-03-14T21:39:49.351 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1' 2024-03-14T21:39:49.351 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.351 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.352 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 48 2024-03-14T21:39:49.353 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30' 2024-03-14T21:39:49.353 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.353 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.354 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 219 2024-03-14T21:39:49.355 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb' 2024-03-14T21:39:49.355 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.355 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.356 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 155 2024-03-14T21:39:49.357 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b' 2024-03-14T21:39:49.357 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.357 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.358 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 242 2024-03-14T21:39:49.358 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2' 2024-03-14T21:39:49.358 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.358 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.359 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 80 2024-03-14T21:39:49.360 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50' 2024-03-14T21:39:49.360 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.360 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.361 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 186 2024-03-14T21:39:49.362 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba' 2024-03-14T21:39:49.362 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.362 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.363 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 210 2024-03-14T21:39:49.363 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2' 2024-03-14T21:39:49.364 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.364 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.365 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 88 2024-03-14T21:39:49.365 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58' 2024-03-14T21:39:49.365 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.365 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.366 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 124 2024-03-14T21:39:49.367 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c' 2024-03-14T21:39:49.367 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.367 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.368 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 175 2024-03-14T21:39:49.369 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf' 2024-03-14T21:39:49.369 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.369 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.370 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 89 2024-03-14T21:39:49.370 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59' 2024-03-14T21:39:49.370 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.370 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.371 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 228 2024-03-14T21:39:49.372 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4' 2024-03-14T21:39:49.372 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.372 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.374 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 34 2024-03-14T21:39:49.375 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22' 2024-03-14T21:39:49.375 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.375 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.376 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 219 2024-03-14T21:39:49.376 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb' 2024-03-14T21:39:49.377 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.377 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.378 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 130 2024-03-14T21:39:49.378 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82' 2024-03-14T21:39:49.378 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.379 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.379 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 128 2024-03-14T21:39:49.380 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80' 2024-03-14T21:39:49.380 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.380 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.381 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 78 2024-03-14T21:39:49.382 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e' 2024-03-14T21:39:49.382 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.382 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.383 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 219 2024-03-14T21:39:49.383 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb' 2024-03-14T21:39:49.383 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.383 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.385 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 51 2024-03-14T21:39:49.385 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33' 2024-03-14T21:39:49.385 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.385 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.386 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 174 2024-03-14T21:39:49.387 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae' 2024-03-14T21:39:49.387 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.387 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.389 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 176 2024-03-14T21:39:49.389 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0' 2024-03-14T21:39:49.389 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.389 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.390 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 45 2024-03-14T21:39:49.390 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d' 2024-03-14T21:39:49.390 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.390 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.391 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 29 2024-03-14T21:39:49.392 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d' 2024-03-14T21:39:49.392 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.392 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.393 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 127 2024-03-14T21:39:49.395 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f' 2024-03-14T21:39:49.395 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.395 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.396 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 119 2024-03-14T21:39:49.396 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77' 2024-03-14T21:39:49.396 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.396 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.397 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 145 2024-03-14T21:39:49.398 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91' 2024-03-14T21:39:49.398 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.398 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.399 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 104 2024-03-14T21:39:49.400 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68' 2024-03-14T21:39:49.400 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.400 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.401 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 156 2024-03-14T21:39:49.401 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c' 2024-03-14T21:39:49.401 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.402 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.403 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 70 2024-03-14T21:39:49.403 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46' 2024-03-14T21:39:49.403 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.403 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.404 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 100 2024-03-14T21:39:49.405 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64' 2024-03-14T21:39:49.405 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.405 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.406 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 162 2024-03-14T21:39:49.407 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2' 2024-03-14T21:39:49.407 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.407 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.408 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 170 2024-03-14T21:39:49.408 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa' 2024-03-14T21:39:49.409 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.409 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.410 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 89 2024-03-14T21:39:49.410 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59' 2024-03-14T21:39:49.410 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.410 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.411 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 151 2024-03-14T21:39:49.412 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97' 2024-03-14T21:39:49.412 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.412 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.413 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 239 2024-03-14T21:39:49.413 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef' 2024-03-14T21:39:49.414 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.414 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.414 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 167 2024-03-14T21:39:49.415 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7' 2024-03-14T21:39:49.415 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.415 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.416 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 182 2024-03-14T21:39:49.417 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7\xb6' 2024-03-14T21:39:49.417 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.417 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.418 INFO:tasks.workunit.client.0.smithi163.stderr:+++ printf %02x 19 2024-03-14T21:39:49.418 INFO:tasks.workunit.client.0.smithi163.stderr:++ raw='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7\xb6\x13' 2024-03-14T21:39:49.419 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i++ )) 2024-03-14T21:39:49.419 INFO:tasks.workunit.client.0.smithi163.stderr:++ (( i < 64 )) 2024-03-14T21:39:49.419 INFO:tasks.workunit.client.0.smithi163.stderr:++ echo '\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7\xb6\x13' 2024-03-14T21:39:49.419 INFO:tasks.workunit.client.0.smithi163.stderr:+ raw_key='\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7\xb6\x13' 2024-03-14T21:39:49.421 INFO:tasks.workunit.client.0.smithi163.stderr:++ _add_enckey fscrypt_test_unlocked_dbench '\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7\xb6\x13' 2024-03-14T21:39:49.422 INFO:tasks.workunit.client.0.smithi163.stderr:++ local mnt=fscrypt_test_unlocked_dbench 2024-03-14T21:39:49.422 INFO:tasks.workunit.client.0.smithi163.stderr:++ local 'raw_key=\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7\xb6\x13' 2024-03-14T21:39:49.422 INFO:tasks.workunit.client.0.smithi163.stderr:++ shift 2 2024-03-14T21:39:49.422 INFO:tasks.workunit.client.0.smithi163.stderr:++ awk '{print $NF}' 2024-03-14T21:39:49.422 INFO:tasks.workunit.client.0.smithi163.stderr:++ echo -ne '\x7d\xe7\x18\xd7\xb4\xf8\x92\x14\x63\xf2\x6d\xf7\x38\x23\xab\xd1\xbc\xd7\xa9\x49\x73\x36\x18\x07\x42\xb1\x30\xdb\x9b\xf2\x50\xba\xd2\x58\x7c\xaf\x59\xe4\x22\xdb\x82\x80\x4e\xdb\x33\xae\xb0\x2d\x1d\x7f\x77\x91\x68\x9c\x46\x64\xa2\xaa\x59\x97\xef\xa7\xb6\x13' 2024-03-14T21:39:49.423 INFO:tasks.workunit.client.0.smithi163.stderr:++ /usr/sbin/xfs_io -c 'add_enckey ' fscrypt_test_unlocked_dbench 2024-03-14T21:39:49.430 INFO:tasks.workunit.client.0.smithi163.stderr:+ keyid=cbbbd0b9853370ada78fb8f8a1f937d3 2024-03-14T21:39:49.431 INFO:tasks.workunit.client.0.smithi163.stderr:+ case ${fscrypt} in 2024-03-14T21:39:49.431 INFO:tasks.workunit.client.0.smithi163.stderr:+ _set_encpolicy fscrypt_test_unlocked_dbench cbbbd0b9853370ada78fb8f8a1f937d3 2024-03-14T21:39:49.431 INFO:tasks.workunit.client.0.smithi163.stderr:+ local dir=fscrypt_test_unlocked_dbench 2024-03-14T21:39:49.431 INFO:tasks.workunit.client.0.smithi163.stderr:+ shift 2024-03-14T21:39:49.431 INFO:tasks.workunit.client.0.smithi163.stderr:+ /usr/sbin/xfs_io -c 'set_encpolicy cbbbd0b9853370ada78fb8f8a1f937d3' fscrypt_test_unlocked_dbench 2024-03-14T21:39:49.437 INFO:tasks.workunit.client.0.smithi163.stderr:+ pushd fscrypt_test_unlocked_dbench 2024-03-14T21:39:49.437 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp/fscrypt_test_unlocked_dbench ~/cephtest/mnt.0/client.0/tmp 2024-03-14T21:39:49.438 INFO:tasks.workunit.client.0.smithi163.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/../suites/dbench.sh 2024-03-14T21:39:49.443 INFO:tasks.workunit.client.0.smithi163.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-14T21:39:49.443 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:39:49.444 INFO:tasks.workunit.client.0.smithi163.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-14T21:39:49.444 INFO:tasks.workunit.client.0.smithi163.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-14T21:39:49.544 INFO:tasks.workunit.client.0.smithi163.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-14T21:39:49.544 INFO:tasks.workunit.client.0.smithi163.stdout:releasing clients 2024-03-14T21:39:50.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 980 21.90 MB/sec warmup 1 sec latency 40.244 ms 2024-03-14T21:39:51.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 3019 16.77 MB/sec warmup 2 sec latency 43.027 ms 2024-03-14T21:39:52.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 4350 13.49 MB/sec warmup 3 sec latency 101.315 ms 2024-03-14T21:39:53.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 5905 11.75 MB/sec warmup 4 sec latency 57.417 ms 2024-03-14T21:39:54.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 7356 11.78 MB/sec warmup 5 sec latency 52.600 ms 2024-03-14T21:39:55.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 8537 10.71 MB/sec warmup 6 sec latency 46.654 ms 2024-03-14T21:39:56.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 10611 10.94 MB/sec warmup 7 sec latency 38.147 ms 2024-03-14T21:39:57.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 11632 9.90 MB/sec warmup 8 sec latency 44.390 ms 2024-03-14T21:39:58.545 INFO:tasks.workunit.client.0.smithi163.stdout: 1 13632 10.10 MB/sec warmup 9 sec latency 45.379 ms 2024-03-14T21:39:59.546 INFO:tasks.workunit.client.0.smithi163.stdout: 1 14918 9.77 MB/sec warmup 10 sec latency 56.089 ms 2024-03-14T21:40:00.546 INFO:tasks.workunit.client.0.smithi163.stdout: 1 16624 9.51 MB/sec warmup 11 sec latency 42.371 ms 2024-03-14T21:40:01.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 18178 9.71 MB/sec warmup 12 sec latency 39.666 ms 2024-03-14T21:40:02.546 INFO:tasks.workunit.client.0.smithi163.stdout: 1 19448 9.37 MB/sec warmup 13 sec latency 42.488 ms 2024-03-14T21:40:03.546 INFO:tasks.workunit.client.0.smithi163.stdout: 1 21542 9.67 MB/sec warmup 14 sec latency 41.506 ms 2024-03-14T21:40:04.546 INFO:tasks.workunit.client.0.smithi163.stdout: 1 22488 9.18 MB/sec warmup 15 sec latency 45.238 ms 2024-03-14T21:40:05.546 INFO:tasks.workunit.client.0.smithi163.stdout: 1 24520 9.32 MB/sec warmup 16 sec latency 45.622 ms 2024-03-14T21:40:06.546 INFO:tasks.workunit.client.0.smithi163.stdout: 1 25798 9.16 MB/sec warmup 17 sec latency 49.624 ms 2024-03-14T21:40:07.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 27793 9.30 MB/sec warmup 18 sec latency 42.084 ms 2024-03-14T21:40:08.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 29217 9.18 MB/sec warmup 19 sec latency 43.888 ms 2024-03-14T21:40:09.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 30908 9.09 MB/sec warmup 20 sec latency 41.347 ms 2024-03-14T21:40:10.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 32437 9.19 MB/sec warmup 21 sec latency 32.713 ms 2024-03-14T21:40:11.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 34311 9.09 MB/sec warmup 22 sec latency 49.964 ms 2024-03-14T21:40:12.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 35829 9.21 MB/sec warmup 23 sec latency 41.875 ms 2024-03-14T21:40:13.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37027 9.05 MB/sec warmup 24 sec latency 47.024 ms 2024-03-14T21:40:14.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37578 8.71 MB/sec warmup 25 sec latency 367.145 ms 2024-03-14T21:40:15.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37600 8.38 MB/sec warmup 26 sec latency 364.395 ms 2024-03-14T21:40:16.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37649 8.07 MB/sec warmup 27 sec latency 364.688 ms 2024-03-14T21:40:17.548 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37705 7.79 MB/sec warmup 28 sec latency 362.604 ms 2024-03-14T21:40:18.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37724 7.52 MB/sec warmup 29 sec latency 362.800 ms 2024-03-14T21:40:19.547 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37779 7.27 MB/sec warmup 30 sec latency 362.718 ms 2024-03-14T21:40:20.548 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37792 7.05 MB/sec warmup 31 sec latency 363.773 ms 2024-03-14T21:40:21.548 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37814 6.84 MB/sec warmup 32 sec latency 724.744 ms 2024-03-14T21:40:22.548 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37836 6.64 MB/sec warmup 33 sec latency 362.419 ms 2024-03-14T21:40:23.548 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37854 6.45 MB/sec warmup 34 sec latency 372.165 ms 2024-03-14T21:40:24.548 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37857 6.26 MB/sec warmup 35 sec latency 1094.557 ms 2024-03-14T21:40:25.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37885 6.09 MB/sec warmup 36 sec latency 363.935 ms 2024-03-14T21:40:26.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37903 5.92 MB/sec warmup 37 sec latency 731.710 ms 2024-03-14T21:40:27.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37910 5.77 MB/sec warmup 38 sec latency 362.449 ms 2024-03-14T21:40:28.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37925 5.62 MB/sec warmup 39 sec latency 372.034 ms 2024-03-14T21:40:29.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37945 5.48 MB/sec warmup 40 sec latency 353.736 ms 2024-03-14T21:40:30.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37950 5.35 MB/sec warmup 41 sec latency 726.720 ms 2024-03-14T21:40:31.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37958 5.22 MB/sec warmup 42 sec latency 362.818 ms 2024-03-14T21:40:32.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37975 5.11 MB/sec warmup 43 sec latency 715.164 ms 2024-03-14T21:40:33.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 37987 5.00 MB/sec warmup 44 sec latency 382.398 ms 2024-03-14T21:40:34.552 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38043 4.89 MB/sec warmup 45 sec latency 367.010 ms 2024-03-14T21:40:35.553 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38103 4.79 MB/sec warmup 46 sec latency 367.881 ms 2024-03-14T21:40:36.553 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38115 4.69 MB/sec warmup 47 sec latency 364.803 ms 2024-03-14T21:40:37.553 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38225 4.65 MB/sec warmup 48 sec latency 364.632 ms 2024-03-14T21:40:38.553 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38257 4.55 MB/sec warmup 49 sec latency 363.618 ms 2024-03-14T21:40:39.553 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38275 4.46 MB/sec warmup 50 sec latency 372.350 ms 2024-03-14T21:40:40.553 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38384 4.38 MB/sec warmup 51 sec latency 361.584 ms 2024-03-14T21:40:41.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38400 4.31 MB/sec warmup 52 sec latency 447.267 ms 2024-03-14T21:40:42.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38406 4.22 MB/sec warmup 53 sec latency 727.961 ms 2024-03-14T21:40:43.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38421 4.16 MB/sec warmup 54 sec latency 623.763 ms 2024-03-14T21:40:44.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38438 4.08 MB/sec warmup 55 sec latency 728.261 ms 2024-03-14T21:40:45.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38477 4.01 MB/sec warmup 56 sec latency 439.138 ms 2024-03-14T21:40:46.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38477 3.94 MB/sec warmup 57 sec latency 1439.184 ms 2024-03-14T21:40:47.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38477 3.87 MB/sec warmup 58 sec latency 2439.211 ms 2024-03-14T21:40:48.583 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38477 3.81 MB/sec warmup 59 sec latency 3439.240 ms 2024-03-14T21:40:49.598 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38477 3.74 MB/sec warmup 60 sec latency 4453.740 ms 2024-03-14T21:40:50.598 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38477 3.68 MB/sec warmup 61 sec latency 5453.858 ms 2024-03-14T21:40:51.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38482 3.62 MB/sec warmup 62 sec latency 5472.469 ms 2024-03-14T21:40:52.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38488 3.56 MB/sec warmup 63 sec latency 729.882 ms 2024-03-14T21:40:53.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38519 3.51 MB/sec warmup 64 sec latency 365.746 ms 2024-03-14T21:40:54.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38701 3.47 MB/sec warmup 65 sec latency 363.777 ms 2024-03-14T21:40:55.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38713 3.41 MB/sec warmup 66 sec latency 750.027 ms 2024-03-14T21:40:56.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38713 3.36 MB/sec warmup 67 sec latency 1385.904 ms 2024-03-14T21:40:57.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38716 3.31 MB/sec warmup 68 sec latency 1993.176 ms 2024-03-14T21:40:58.601 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38716 3.27 MB/sec warmup 69 sec latency 1392.797 ms 2024-03-14T21:40:59.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38716 3.22 MB/sec warmup 70 sec latency 2392.899 ms 2024-03-14T21:41:00.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38716 3.17 MB/sec warmup 71 sec latency 3392.977 ms 2024-03-14T21:41:01.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 38716 3.13 MB/sec warmup 72 sec latency 4393.072 ms 2024-03-14T21:41:02.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 39315 3.16 MB/sec warmup 73 sec latency 4778.394 ms 2024-03-14T21:41:03.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 39327 3.12 MB/sec warmup 74 sec latency 369.516 ms 2024-03-14T21:41:04.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 40210 3.11 MB/sec warmup 75 sec latency 362.997 ms 2024-03-14T21:41:05.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 42427 3.23 MB/sec warmup 76 sec latency 38.471 ms 2024-03-14T21:41:06.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 43481 3.26 MB/sec warmup 77 sec latency 46.395 ms 2024-03-14T21:41:07.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 44085 3.27 MB/sec warmup 78 sec latency 621.035 ms 2024-03-14T21:41:08.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 44085 3.23 MB/sec warmup 79 sec latency 1621.145 ms 2024-03-14T21:41:09.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 44085 3.19 MB/sec warmup 80 sec latency 2621.250 ms 2024-03-14T21:41:10.602 INFO:tasks.workunit.client.0.smithi163.stdout: 1 44085 3.15 MB/sec warmup 81 sec latency 3621.353 ms 2024-03-14T21:41:11.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 44085 3.11 MB/sec warmup 82 sec latency 4621.445 ms 2024-03-14T21:41:12.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 45156 3.11 MB/sec warmup 83 sec latency 5002.673 ms 2024-03-14T21:41:13.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 46616 3.21 MB/sec warmup 84 sec latency 14.471 ms 2024-03-14T21:41:14.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 47273 3.19 MB/sec warmup 85 sec latency 366.996 ms 2024-03-14T21:41:15.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 49143 3.28 MB/sec warmup 86 sec latency 45.068 ms 2024-03-14T21:41:16.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 50421 3.32 MB/sec warmup 87 sec latency 46.608 ms 2024-03-14T21:41:17.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 51299 3.34 MB/sec warmup 88 sec latency 383.880 ms 2024-03-14T21:41:18.603 INFO:tasks.workunit.client.0.smithi163.stdout: 1 51299 3.30 MB/sec warmup 89 sec latency 1383.970 ms 2024-03-14T21:41:19.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 51299 3.26 MB/sec warmup 90 sec latency 2384.052 ms 2024-03-14T21:41:20.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 51299 3.23 MB/sec warmup 91 sec latency 3384.134 ms 2024-03-14T21:41:21.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 51299 3.19 MB/sec warmup 92 sec latency 4384.215 ms 2024-03-14T21:41:21.664 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:41:21.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:41:21.670 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:41:22.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 52086 3.18 MB/sec warmup 93 sec latency 5003.500 ms 2024-03-14T21:41:23.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 53577 3.27 MB/sec warmup 94 sec latency 43.827 ms 2024-03-14T21:41:24.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 54621 3.29 MB/sec warmup 95 sec latency 50.906 ms 2024-03-14T21:41:25.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 56593 3.36 MB/sec warmup 96 sec latency 39.858 ms 2024-03-14T21:41:26.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 57704 3.38 MB/sec warmup 97 sec latency 45.131 ms 2024-03-14T21:41:27.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 59765 3.46 MB/sec warmup 98 sec latency 48.650 ms 2024-03-14T21:41:28.604 INFO:tasks.workunit.client.0.smithi163.stdout: 1 60552 3.49 MB/sec warmup 99 sec latency 635.038 ms 2024-03-14T21:41:29.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 60552 3.46 MB/sec warmup 100 sec latency 1635.119 ms 2024-03-14T21:41:30.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 60552 3.42 MB/sec warmup 101 sec latency 2635.218 ms 2024-03-14T21:41:31.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 60552 3.39 MB/sec warmup 102 sec latency 3635.333 ms 2024-03-14T21:41:32.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 60552 3.35 MB/sec warmup 103 sec latency 4635.415 ms 2024-03-14T21:41:33.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 61050 3.33 MB/sec warmup 104 sec latency 5005.390 ms 2024-03-14T21:41:34.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62759 3.36 MB/sec warmup 105 sec latency 48.237 ms 2024-03-14T21:41:35.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62794 3.33 MB/sec warmup 106 sec latency 367.079 ms 2024-03-14T21:41:36.605 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62797 3.30 MB/sec warmup 107 sec latency 515.648 ms 2024-03-14T21:41:37.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62808 3.27 MB/sec warmup 108 sec latency 726.863 ms 2024-03-14T21:41:38.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62810 3.24 MB/sec warmup 109 sec latency 746.052 ms 2024-03-14T21:41:39.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62820 3.21 MB/sec warmup 110 sec latency 632.934 ms 2024-03-14T21:41:40.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62820 3.19 MB/sec warmup 111 sec latency 1632.997 ms 2024-03-14T21:41:41.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62820 3.16 MB/sec warmup 112 sec latency 2633.065 ms 2024-03-14T21:41:42.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 62820 3.13 MB/sec warmup 113 sec latency 3633.167 ms 2024-03-14T21:41:43.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 63596 3.16 MB/sec warmup 114 sec latency 4292.470 ms 2024-03-14T21:41:44.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 64670 3.18 MB/sec warmup 115 sec latency 65.175 ms 2024-03-14T21:41:45.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 66439 3.21 MB/sec warmup 116 sec latency 43.739 ms 2024-03-14T21:41:46.628 INFO:tasks.workunit.client.0.smithi163.stdout: 1 67669 3.28 MB/sec warmup 117 sec latency 89.855 ms 2024-03-14T21:41:47.629 INFO:tasks.workunit.client.0.smithi163.stdout: 1 68410 3.26 MB/sec warmup 118 sec latency 61.743 ms 2024-03-14T21:41:48.629 INFO:tasks.workunit.client.0.smithi163.stdout: 1 70429 3.33 MB/sec warmup 119 sec latency 42.800 ms 2024-03-14T21:41:50.629 INFO:tasks.workunit.client.0.smithi163.stdout: 1 71121 0.00 MB/sec execute 1 sec latency 1664.483 ms 2024-03-14T21:41:51.629 INFO:tasks.workunit.client.0.smithi163.stdout: 1 71121 0.00 MB/sec execute 2 sec latency 2664.566 ms 2024-03-14T21:41:52.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:41:52.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:41:52.287 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:41:52.629 INFO:tasks.workunit.client.0.smithi163.stdout: 1 71121 0.00 MB/sec execute 3 sec latency 3664.667 ms 2024-03-14T21:41:53.629 INFO:tasks.workunit.client.0.smithi163.stdout: 1 71121 0.00 MB/sec execute 4 sec latency 4664.767 ms 2024-03-14T21:41:54.629 INFO:tasks.workunit.client.0.smithi163.stdout: 1 71646 0.30 MB/sec execute 5 sec latency 5014.183 ms 2024-03-14T21:41:55.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 73392 1.40 MB/sec execute 6 sec latency 44.752 ms 2024-03-14T21:41:56.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 74290 2.25 MB/sec execute 7 sec latency 286.060 ms 2024-03-14T21:41:57.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 75018 2.56 MB/sec execute 8 sec latency 611.588 ms 2024-03-14T21:41:58.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 76600 2.90 MB/sec execute 9 sec latency 44.123 ms 2024-03-14T21:41:59.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77557 3.31 MB/sec execute 10 sec latency 423.086 ms 2024-03-14T21:42:00.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77557 3.01 MB/sec execute 11 sec latency 1423.175 ms 2024-03-14T21:42:01.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77557 2.75 MB/sec execute 12 sec latency 2423.258 ms 2024-03-14T21:42:02.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77557 2.54 MB/sec execute 13 sec latency 3423.347 ms 2024-03-14T21:42:03.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 77557 2.36 MB/sec execute 14 sec latency 4423.436 ms 2024-03-14T21:42:04.630 INFO:tasks.workunit.client.0.smithi163.stdout: 1 78298 2.62 MB/sec execute 15 sec latency 5025.360 ms 2024-03-14T21:42:05.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 79076 2.53 MB/sec execute 16 sec latency 57.839 ms 2024-03-14T21:42:06.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 80963 2.99 MB/sec execute 17 sec latency 92.533 ms 2024-03-14T21:42:07.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 82100 3.26 MB/sec execute 18 sec latency 26.607 ms 2024-03-14T21:42:08.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 83654 3.38 MB/sec execute 19 sec latency 47.616 ms 2024-03-14T21:42:09.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 85452 3.88 MB/sec execute 20 sec latency 43.428 ms 2024-03-14T21:42:10.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 86060 3.74 MB/sec execute 21 sec latency 297.413 ms 2024-03-14T21:42:11.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 86060 3.57 MB/sec execute 22 sec latency 1297.511 ms 2024-03-14T21:42:12.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 86060 3.41 MB/sec execute 23 sec latency 2297.594 ms 2024-03-14T21:42:13.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 86060 3.27 MB/sec execute 24 sec latency 3297.675 ms 2024-03-14T21:42:14.631 INFO:tasks.workunit.client.0.smithi163.stdout: 1 86060 3.14 MB/sec execute 25 sec latency 4297.761 ms 2024-03-14T21:42:15.632 INFO:tasks.workunit.client.0.smithi163.stdout: 1 87523 3.26 MB/sec execute 26 sec latency 4625.099 ms 2024-03-14T21:42:16.632 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89044 3.58 MB/sec execute 27 sec latency 49.020 ms 2024-03-14T21:42:17.635 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89055 3.45 MB/sec execute 28 sec latency 363.777 ms 2024-03-14T21:42:18.635 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89086 3.34 MB/sec execute 29 sec latency 365.978 ms 2024-03-14T21:42:19.635 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89089 3.23 MB/sec execute 30 sec latency 724.894 ms 2024-03-14T21:42:20.642 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89093 3.13 MB/sec execute 31 sec latency 739.994 ms 2024-03-14T21:42:21.642 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89093 3.03 MB/sec execute 32 sec latency 1740.024 ms 2024-03-14T21:42:22.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:42:22.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:42:22.408 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:42:22.459 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T21:42:22.481+0000 7f137ed27640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46475) UID: 0 2024-03-14T21:42:22.459 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-03-14T21:42:22.481+0000 7f5aad84b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 46475) UID: 0 2024-03-14T21:42:22.642 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89093 2.94 MB/sec execute 33 sec latency 2740.091 ms 2024-03-14T21:42:23.642 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89093 2.85 MB/sec execute 34 sec latency 3740.193 ms 2024-03-14T21:42:24.642 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89093 2.77 MB/sec execute 35 sec latency 4740.300 ms 2024-03-14T21:42:25.642 INFO:tasks.workunit.client.0.smithi163.stdout: 1 89187 2.69 MB/sec execute 36 sec latency 5362.619 ms 2024-03-14T21:42:26.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 90234 2.76 MB/sec execute 37 sec latency 63.444 ms 2024-03-14T21:42:27.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 92065 2.93 MB/sec execute 38 sec latency 51.731 ms 2024-03-14T21:42:28.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 92809 2.98 MB/sec execute 39 sec latency 69.475 ms 2024-03-14T21:42:29.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 94530 3.05 MB/sec execute 40 sec latency 49.959 ms 2024-03-14T21:42:30.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 96138 3.29 MB/sec execute 41 sec latency 53.569 ms 2024-03-14T21:42:31.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 96436 3.22 MB/sec execute 42 sec latency 673.098 ms 2024-03-14T21:42:32.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 96436 3.14 MB/sec execute 43 sec latency 1673.215 ms 2024-03-14T21:42:33.643 INFO:tasks.workunit.client.0.smithi163.stdout: 1 96436 3.07 MB/sec execute 44 sec latency 2673.324 ms 2024-03-14T21:42:34.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 96436 3.00 MB/sec execute 45 sec latency 3673.474 ms 2024-03-14T21:42:35.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 96436 2.94 MB/sec execute 46 sec latency 4673.579 ms 2024-03-14T21:42:36.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 97309 2.98 MB/sec execute 47 sec latency 4995.881 ms 2024-03-14T21:42:37.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 99565 3.21 MB/sec execute 48 sec latency 36.546 ms 2024-03-14T21:42:38.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 100050 3.15 MB/sec execute 49 sec latency 415.788 ms 2024-03-14T21:42:39.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 101778 3.23 MB/sec execute 50 sec latency 64.647 ms 2024-03-14T21:42:40.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 103354 3.40 MB/sec execute 51 sec latency 42.865 ms 2024-03-14T21:42:41.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 103846 3.35 MB/sec execute 52 sec latency 452.765 ms 2024-03-14T21:42:42.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 103846 3.29 MB/sec execute 53 sec latency 1452.843 ms 2024-03-14T21:42:43.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 103846 3.23 MB/sec execute 54 sec latency 2452.925 ms 2024-03-14T21:42:44.644 INFO:tasks.workunit.client.0.smithi163.stdout: 1 103846 3.17 MB/sec execute 55 sec latency 3453.047 ms 2024-03-14T21:42:45.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 103846 3.11 MB/sec execute 56 sec latency 4453.124 ms 2024-03-14T21:42:46.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 104602 3.13 MB/sec execute 57 sec latency 5004.318 ms 2024-03-14T21:42:47.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 106680 3.31 MB/sec execute 58 sec latency 41.551 ms 2024-03-14T21:42:48.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 107253 3.27 MB/sec execute 59 sec latency 261.709 ms 2024-03-14T21:42:49.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 108828 3.33 MB/sec execute 60 sec latency 342.438 ms 2024-03-14T21:42:50.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 110380 3.47 MB/sec execute 61 sec latency 39.232 ms 2024-03-14T21:42:51.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 111519 3.50 MB/sec execute 62 sec latency 54.375 ms 2024-03-14T21:42:52.645 INFO:tasks.workunit.client.0.smithi163.stdout: 1 112377 3.47 MB/sec execute 63 sec latency 713.451 ms 2024-03-14T21:42:53.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 112377 3.42 MB/sec execute 64 sec latency 1713.533 ms 2024-03-14T21:42:54.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 112377 3.36 MB/sec execute 65 sec latency 2713.616 ms 2024-03-14T21:42:55.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 112377 3.31 MB/sec execute 66 sec latency 3713.694 ms 2024-03-14T21:42:56.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 112377 3.26 MB/sec execute 67 sec latency 4713.801 ms 2024-03-14T21:42:57.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 113524 3.37 MB/sec execute 68 sec latency 5010.527 ms 2024-03-14T21:42:58.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114428 3.35 MB/sec execute 69 sec latency 73.754 ms 2024-03-14T21:42:59.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114446 3.31 MB/sec execute 70 sec latency 353.759 ms 2024-03-14T21:43:00.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114456 3.26 MB/sec execute 71 sec latency 725.721 ms 2024-03-14T21:43:01.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114458 3.22 MB/sec execute 72 sec latency 727.839 ms 2024-03-14T21:43:02.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114467 3.17 MB/sec execute 73 sec latency 362.514 ms 2024-03-14T21:43:03.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114470 3.13 MB/sec execute 74 sec latency 728.616 ms 2024-03-14T21:43:04.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114480 3.09 MB/sec execute 75 sec latency 594.102 ms 2024-03-14T21:43:05.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114480 3.05 MB/sec execute 76 sec latency 1594.189 ms 2024-03-14T21:43:06.646 INFO:tasks.workunit.client.0.smithi163.stdout: 1 114480 3.01 MB/sec execute 77 sec latency 2594.269 ms 2024-03-14T21:43:07.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 115127 3.03 MB/sec execute 78 sec latency 3150.829 ms 2024-03-14T21:43:08.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 117246 3.15 MB/sec execute 79 sec latency 45.539 ms 2024-03-14T21:43:09.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 118079 3.14 MB/sec execute 80 sec latency 50.881 ms 2024-03-14T21:43:10.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 119437 3.17 MB/sec execute 81 sec latency 98.700 ms 2024-03-14T21:43:11.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 120974 3.28 MB/sec execute 82 sec latency 51.168 ms 2024-03-14T21:43:12.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 122228 3.30 MB/sec execute 83 sec latency 47.739 ms 2024-03-14T21:43:13.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 123010 3.29 MB/sec execute 84 sec latency 736.308 ms 2024-03-14T21:43:14.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 123010 3.25 MB/sec execute 85 sec latency 1736.385 ms 2024-03-14T21:43:15.647 INFO:tasks.workunit.client.0.smithi163.stdout: 1 123010 3.21 MB/sec execute 86 sec latency 2736.468 ms 2024-03-14T21:43:16.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 123010 3.17 MB/sec execute 87 sec latency 3736.546 ms 2024-03-14T21:43:17.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 123010 3.14 MB/sec execute 88 sec latency 4736.632 ms 2024-03-14T21:43:18.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 124242 3.22 MB/sec execute 89 sec latency 5024.363 ms 2024-03-14T21:43:19.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 125096 3.21 MB/sec execute 90 sec latency 66.185 ms 2024-03-14T21:43:20.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 126166 3.23 MB/sec execute 91 sec latency 171.989 ms 2024-03-14T21:43:21.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 127791 3.32 MB/sec execute 92 sec latency 66.942 ms 2024-03-14T21:43:22.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 128717 3.31 MB/sec execute 93 sec latency 39.666 ms 2024-03-14T21:43:23.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 130074 3.34 MB/sec execute 94 sec latency 492.510 ms 2024-03-14T21:43:24.648 INFO:tasks.workunit.client.0.smithi163.stdout: 1 130074 3.31 MB/sec execute 95 sec latency 1492.592 ms 2024-03-14T21:43:25.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 130074 3.27 MB/sec execute 96 sec latency 2492.675 ms 2024-03-14T21:43:26.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 130074 3.24 MB/sec execute 97 sec latency 3492.764 ms 2024-03-14T21:43:27.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 130074 3.21 MB/sec execute 98 sec latency 4492.841 ms 2024-03-14T21:43:28.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 130719 3.23 MB/sec execute 99 sec latency 5018.461 ms 2024-03-14T21:43:29.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 132077 3.27 MB/sec execute 100 sec latency 60.875 ms 2024-03-14T21:43:30.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 133707 3.30 MB/sec execute 101 sec latency 75.202 ms 2024-03-14T21:43:31.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 135206 3.39 MB/sec execute 102 sec latency 208.054 ms 2024-03-14T21:43:32.649 INFO:tasks.workunit.client.0.smithi163.stdout: 1 136441 3.40 MB/sec execute 103 sec latency 65.039 ms 2024-03-14T21:43:33.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 138529 3.49 MB/sec execute 104 sec latency 45.524 ms 2024-03-14T21:43:34.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 138741 3.47 MB/sec execute 105 sec latency 728.453 ms 2024-03-14T21:43:35.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 138741 3.44 MB/sec execute 106 sec latency 1728.541 ms 2024-03-14T21:43:36.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 138741 3.41 MB/sec execute 107 sec latency 2728.658 ms 2024-03-14T21:43:37.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 138741 3.38 MB/sec execute 108 sec latency 3728.778 ms 2024-03-14T21:43:38.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 138741 3.35 MB/sec execute 109 sec latency 4728.897 ms 2024-03-14T21:43:39.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 139311 3.32 MB/sec execute 110 sec latency 5012.066 ms 2024-03-14T21:43:40.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141362 3.40 MB/sec execute 111 sec latency 49.966 ms 2024-03-14T21:43:41.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141385 3.37 MB/sec execute 112 sec latency 350.566 ms 2024-03-14T21:43:42.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141391 3.34 MB/sec execute 113 sec latency 603.033 ms 2024-03-14T21:43:43.650 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141535 3.31 MB/sec execute 114 sec latency 723.985 ms 2024-03-14T21:43:44.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141542 3.29 MB/sec execute 115 sec latency 425.585 ms 2024-03-14T21:43:45.658 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141542 3.26 MB/sec execute 116 sec latency 1433.009 ms 2024-03-14T21:43:46.658 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141542 3.23 MB/sec execute 117 sec latency 2433.073 ms 2024-03-14T21:43:47.658 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141542 3.20 MB/sec execute 118 sec latency 3433.162 ms 2024-03-14T21:43:48.658 INFO:tasks.workunit.client.0.smithi163.stdout: 1 141542 3.17 MB/sec execute 119 sec latency 4433.265 ms 2024-03-14T21:43:49.658 INFO:tasks.workunit.client.0.smithi163.stdout: 1 142134 3.20 MB/sec execute 120 sec latency 4995.741 ms 2024-03-14T21:43:50.658 INFO:tasks.workunit.client.0.smithi163.stdout: 1 142957 3.19 MB/sec execute 121 sec latency 60.623 ms 2024-03-14T21:43:51.658 INFO:tasks.workunit.client.0.smithi163.stdout: 1 144889 3.25 MB/sec execute 122 sec latency 59.940 ms 2024-03-14T21:43:52.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 145766 3.27 MB/sec execute 123 sec latency 100.578 ms 2024-03-14T21:43:53.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 146643 3.27 MB/sec execute 124 sec latency 53.228 ms 2024-03-14T21:43:54.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 148473 3.32 MB/sec execute 125 sec latency 47.371 ms 2024-03-14T21:43:55.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 149740 3.35 MB/sec execute 126 sec latency 68.567 ms 2024-03-14T21:43:56.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 151436 3.38 MB/sec execute 127 sec latency 52.134 ms 2024-03-14T21:43:57.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 152935 3.45 MB/sec execute 128 sec latency 49.218 ms 2024-03-14T21:43:58.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 153487 3.43 MB/sec execute 129 sec latency 378.495 ms 2024-03-14T21:43:59.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 153487 3.40 MB/sec execute 130 sec latency 1378.569 ms 2024-03-14T21:44:00.659 INFO:tasks.workunit.client.0.smithi163.stdout: 1 155004 3.42 MB/sec execute 131 sec latency 1637.557 ms 2024-03-14T21:44:01.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 156552 3.49 MB/sec execute 132 sec latency 44.282 ms 2024-03-14T21:44:02.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 157016 3.47 MB/sec execute 133 sec latency 229.652 ms 2024-03-14T21:44:03.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 158440 3.49 MB/sec execute 134 sec latency 257.796 ms 2024-03-14T21:44:04.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 159981 3.55 MB/sec execute 135 sec latency 54.547 ms 2024-03-14T21:44:05.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 160420 3.53 MB/sec execute 136 sec latency 506.685 ms 2024-03-14T21:44:06.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 160420 3.50 MB/sec execute 137 sec latency 1506.791 ms 2024-03-14T21:44:07.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 160420 3.48 MB/sec execute 138 sec latency 2506.873 ms 2024-03-14T21:44:08.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 160420 3.45 MB/sec execute 139 sec latency 3506.982 ms 2024-03-14T21:44:09.660 INFO:tasks.workunit.client.0.smithi163.stdout: 1 160420 3.43 MB/sec execute 140 sec latency 4507.053 ms 2024-03-14T21:44:10.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 161074 3.44 MB/sec execute 141 sec latency 5005.073 ms 2024-03-14T21:44:11.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 163036 3.48 MB/sec execute 142 sec latency 48.149 ms 2024-03-14T21:44:12.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 163973 3.49 MB/sec execute 143 sec latency 153.512 ms 2024-03-14T21:44:13.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 165560 3.51 MB/sec execute 144 sec latency 66.273 ms 2024-03-14T21:44:14.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 167039 3.57 MB/sec execute 145 sec latency 64.432 ms 2024-03-14T21:44:15.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 167983 3.56 MB/sec execute 146 sec latency 58.000 ms 2024-03-14T21:44:16.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 168562 3.56 MB/sec execute 147 sec latency 751.166 ms 2024-03-14T21:44:17.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 168562 3.53 MB/sec execute 148 sec latency 1751.272 ms 2024-03-14T21:44:18.661 INFO:tasks.workunit.client.0.smithi163.stdout: 1 168562 3.51 MB/sec execute 149 sec latency 2751.352 ms 2024-03-14T21:44:19.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 168562 3.49 MB/sec execute 150 sec latency 3751.423 ms 2024-03-14T21:44:20.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 168562 3.46 MB/sec execute 151 sec latency 4751.495 ms 2024-03-14T21:44:21.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 169938 3.49 MB/sec execute 152 sec latency 5000.540 ms 2024-03-14T21:44:22.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170949 3.51 MB/sec execute 153 sec latency 130.199 ms 2024-03-14T21:44:23.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170953 3.49 MB/sec execute 154 sec latency 411.850 ms 2024-03-14T21:44:24.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170957 3.46 MB/sec execute 155 sec latency 363.891 ms 2024-03-14T21:44:25.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170981 3.44 MB/sec execute 156 sec latency 363.695 ms 2024-03-14T21:44:26.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170986 3.42 MB/sec execute 157 sec latency 725.371 ms 2024-03-14T21:44:27.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170994 3.40 MB/sec execute 158 sec latency 362.335 ms 2024-03-14T21:44:28.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170996 3.38 MB/sec execute 159 sec latency 363.043 ms 2024-03-14T21:44:29.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170996 3.36 MB/sec execute 160 sec latency 1278.528 ms 2024-03-14T21:44:30.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 170996 3.33 MB/sec execute 161 sec latency 2278.562 ms 2024-03-14T21:44:31.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 171005 3.31 MB/sec execute 162 sec latency 2834.249 ms 2024-03-14T21:44:32.662 INFO:tasks.workunit.client.0.smithi163.stdout: 1 172473 3.33 MB/sec execute 163 sec latency 517.789 ms 2024-03-14T21:44:33.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 174059 3.39 MB/sec execute 164 sec latency 46.106 ms 2024-03-14T21:44:34.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 174506 3.37 MB/sec execute 165 sec latency 105.045 ms 2024-03-14T21:44:35.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 175651 3.38 MB/sec execute 166 sec latency 149.714 ms 2024-03-14T21:44:36.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 177599 3.44 MB/sec execute 167 sec latency 48.044 ms 2024-03-14T21:44:37.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 177755 3.42 MB/sec execute 168 sec latency 764.480 ms 2024-03-14T21:44:38.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 177755 3.40 MB/sec execute 169 sec latency 1764.516 ms 2024-03-14T21:44:39.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 177755 3.38 MB/sec execute 170 sec latency 2764.552 ms 2024-03-14T21:44:40.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 177755 3.36 MB/sec execute 171 sec latency 3764.588 ms 2024-03-14T21:44:41.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 177755 3.34 MB/sec execute 172 sec latency 4764.617 ms 2024-03-14T21:44:42.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 178415 3.33 MB/sec execute 173 sec latency 5045.279 ms 2024-03-14T21:44:43.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 180632 3.39 MB/sec execute 174 sec latency 41.755 ms 2024-03-14T21:44:44.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 181589 3.40 MB/sec execute 175 sec latency 89.522 ms 2024-03-14T21:44:45.663 INFO:tasks.workunit.client.0.smithi163.stdout: 1 182609 3.40 MB/sec execute 176 sec latency 162.977 ms 2024-03-14T21:44:46.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 184704 3.46 MB/sec execute 177 sec latency 48.173 ms 2024-03-14T21:44:47.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 185132 3.45 MB/sec execute 178 sec latency 545.405 ms 2024-03-14T21:44:48.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 185132 3.43 MB/sec execute 179 sec latency 1545.456 ms 2024-03-14T21:44:49.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 185132 3.41 MB/sec execute 180 sec latency 2545.503 ms 2024-03-14T21:44:50.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 185132 3.39 MB/sec execute 181 sec latency 3545.548 ms 2024-03-14T21:44:51.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 185132 3.37 MB/sec execute 182 sec latency 4545.575 ms 2024-03-14T21:44:52.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 185774 3.37 MB/sec execute 183 sec latency 5024.301 ms 2024-03-14T21:44:53.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 187866 3.41 MB/sec execute 184 sec latency 48.681 ms 2024-03-14T21:44:54.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 188769 3.42 MB/sec execute 185 sec latency 129.931 ms 2024-03-14T21:44:55.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 190189 3.43 MB/sec execute 186 sec latency 123.688 ms 2024-03-14T21:44:56.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 191796 3.48 MB/sec execute 187 sec latency 49.309 ms 2024-03-14T21:44:57.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 192589 3.47 MB/sec execute 188 sec latency 47.753 ms 2024-03-14T21:44:58.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 193053 3.47 MB/sec execute 189 sec latency 773.934 ms 2024-03-14T21:44:59.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 193053 3.46 MB/sec execute 190 sec latency 1773.968 ms 2024-03-14T21:45:00.665 INFO:tasks.workunit.client.0.smithi163.stdout: 1 193053 3.44 MB/sec execute 191 sec latency 2773.997 ms 2024-03-14T21:45:01.664 INFO:tasks.workunit.client.0.smithi163.stdout: 1 193053 3.42 MB/sec execute 192 sec latency 3774.028 ms 2024-03-14T21:45:02.665 INFO:tasks.workunit.client.0.smithi163.stdout: 1 193053 3.40 MB/sec execute 193 sec latency 4774.057 ms 2024-03-14T21:45:03.665 INFO:tasks.workunit.client.0.smithi163.stdout: 1 194434 3.42 MB/sec execute 194 sec latency 5008.436 ms 2024-03-14T21:45:04.665 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195489 3.44 MB/sec execute 195 sec latency 115.903 ms 2024-03-14T21:45:05.665 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195496 3.42 MB/sec execute 196 sec latency 370.995 ms 2024-03-14T21:45:06.665 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195503 3.40 MB/sec execute 197 sec latency 367.501 ms 2024-03-14T21:45:06.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:45:06.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:45:06.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:45:07.665 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195504 3.39 MB/sec execute 198 sec latency 926.582 ms 2024-03-14T21:45:08.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195520 3.37 MB/sec execute 199 sec latency 1092.429 ms 2024-03-14T21:45:09.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195524 3.35 MB/sec execute 200 sec latency 734.825 ms 2024-03-14T21:45:10.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195536 3.34 MB/sec execute 201 sec latency 362.304 ms 2024-03-14T21:45:11.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195536 3.32 MB/sec execute 202 sec latency 1272.249 ms 2024-03-14T21:45:12.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195536 3.30 MB/sec execute 203 sec latency 2272.296 ms 2024-03-14T21:45:13.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 195897 3.29 MB/sec execute 204 sec latency 2814.376 ms 2024-03-14T21:45:14.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 197513 3.31 MB/sec execute 205 sec latency 47.488 ms 2024-03-14T21:45:15.670 INFO:tasks.workunit.client.0.smithi163.stdout: 1 199000 3.35 MB/sec execute 206 sec latency 40.681 ms 2024-03-14T21:45:16.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 199522 3.33 MB/sec execute 207 sec latency 137.691 ms 2024-03-14T21:45:17.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 201088 3.35 MB/sec execute 208 sec latency 52.621 ms 2024-03-14T21:45:18.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 202585 3.39 MB/sec execute 209 sec latency 43.976 ms 2024-03-14T21:45:19.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 202768 3.37 MB/sec execute 210 sec latency 785.467 ms 2024-03-14T21:45:20.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 202768 3.36 MB/sec execute 211 sec latency 1785.498 ms 2024-03-14T21:45:21.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 202768 3.34 MB/sec execute 212 sec latency 2785.534 ms 2024-03-14T21:45:22.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 202768 3.33 MB/sec execute 213 sec latency 3785.575 ms 2024-03-14T21:45:23.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 202768 3.31 MB/sec execute 214 sec latency 4785.605 ms 2024-03-14T21:45:24.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 203611 3.32 MB/sec execute 215 sec latency 5000.746 ms 2024-03-14T21:45:25.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 205394 3.34 MB/sec execute 216 sec latency 52.232 ms 2024-03-14T21:45:26.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 206207 3.35 MB/sec execute 217 sec latency 86.989 ms 2024-03-14T21:45:27.671 INFO:tasks.workunit.client.0.smithi163.stdout: 1 207414 3.36 MB/sec execute 218 sec latency 48.998 ms 2024-03-14T21:45:28.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 209347 3.40 MB/sec execute 219 sec latency 44.190 ms 2024-03-14T21:45:29.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 209704 3.40 MB/sec execute 220 sec latency 545.461 ms 2024-03-14T21:45:30.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 209704 3.38 MB/sec execute 221 sec latency 1545.494 ms 2024-03-14T21:45:31.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 209704 3.36 MB/sec execute 222 sec latency 2545.530 ms 2024-03-14T21:45:32.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 209704 3.35 MB/sec execute 223 sec latency 3545.567 ms 2024-03-14T21:45:33.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 209704 3.33 MB/sec execute 224 sec latency 4545.595 ms 2024-03-14T21:45:34.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 210162 3.32 MB/sec execute 225 sec latency 5025.571 ms 2024-03-14T21:45:35.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 211733 3.34 MB/sec execute 226 sec latency 87.223 ms 2024-03-14T21:45:36.672 INFO:tasks.workunit.client.0.smithi163.stdout: 1 213131 3.37 MB/sec execute 227 sec latency 57.003 ms 2024-03-14T21:45:37.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:45:37.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:45:37.037 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:45:37.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 214007 3.37 MB/sec execute 228 sec latency 52.509 ms 2024-03-14T21:45:38.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 216069 3.40 MB/sec execute 229 sec latency 52.205 ms 2024-03-14T21:45:39.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 217279 3.42 MB/sec execute 230 sec latency 50.375 ms 2024-03-14T21:45:40.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 217463 3.41 MB/sec execute 231 sec latency 794.773 ms 2024-03-14T21:45:41.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 217463 3.39 MB/sec execute 232 sec latency 1794.806 ms 2024-03-14T21:45:42.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 217463 3.38 MB/sec execute 233 sec latency 2794.863 ms 2024-03-14T21:45:43.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 217463 3.36 MB/sec execute 234 sec latency 3794.908 ms 2024-03-14T21:45:44.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 217463 3.35 MB/sec execute 235 sec latency 4794.949 ms 2024-03-14T21:45:45.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 218998 3.36 MB/sec execute 236 sec latency 5004.982 ms 2024-03-14T21:45:46.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220391 3.39 MB/sec execute 237 sec latency 138.722 ms 2024-03-14T21:45:47.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220403 3.38 MB/sec execute 238 sec latency 366.941 ms 2024-03-14T21:45:48.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220409 3.37 MB/sec execute 239 sec latency 364.677 ms 2024-03-14T21:45:49.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220414 3.35 MB/sec execute 240 sec latency 725.271 ms 2024-03-14T21:45:50.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220422 3.34 MB/sec execute 241 sec latency 722.791 ms 2024-03-14T21:45:51.673 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220430 3.32 MB/sec execute 242 sec latency 403.782 ms 2024-03-14T21:45:52.674 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220442 3.31 MB/sec execute 243 sec latency 740.076 ms 2024-03-14T21:45:53.674 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220442 3.30 MB/sec execute 244 sec latency 1275.853 ms 2024-03-14T21:45:54.674 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220442 3.28 MB/sec execute 245 sec latency 2275.953 ms 2024-03-14T21:45:55.674 INFO:tasks.workunit.client.0.smithi163.stdout: 1 220831 3.27 MB/sec execute 246 sec latency 2823.032 ms 2024-03-14T21:45:56.674 INFO:tasks.workunit.client.0.smithi163.stdout: 1 222383 3.29 MB/sec execute 247 sec latency 74.482 ms 2024-03-14T21:45:57.674 INFO:tasks.workunit.client.0.smithi163.stdout: 1 223934 3.32 MB/sec execute 248 sec latency 48.394 ms 2024-03-14T21:45:58.674 INFO:tasks.workunit.client.0.smithi163.stdout: 1 224314 3.31 MB/sec execute 249 sec latency 293.598 ms 2024-03-14T21:45:59.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 225992 3.32 MB/sec execute 250 sec latency 108.797 ms 2024-03-14T21:46:00.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 227647 3.36 MB/sec execute 251 sec latency 45.040 ms 2024-03-14T21:46:01.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 228946 3.37 MB/sec execute 252 sec latency 181.106 ms 2024-03-14T21:46:02.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 228946 3.35 MB/sec execute 253 sec latency 1181.220 ms 2024-03-14T21:46:03.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 228946 3.34 MB/sec execute 254 sec latency 2181.317 ms 2024-03-14T21:46:04.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 228946 3.33 MB/sec execute 255 sec latency 3181.404 ms 2024-03-14T21:46:05.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 228946 3.31 MB/sec execute 256 sec latency 4181.490 ms 2024-03-14T21:46:06.678 INFO:tasks.workunit.client.0.smithi163.stdout: 1 230405 3.33 MB/sec execute 257 sec latency 4376.390 ms 2024-03-14T21:46:07.334 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:46:07.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:46:07.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:46:07.675 INFO:tasks.workunit.client.0.smithi163.stdout: 1 231528 3.34 MB/sec execute 258 sec latency 41.669 ms 2024-03-14T21:46:08.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 232641 3.35 MB/sec execute 259 sec latency 313.149 ms 2024-03-14T21:46:09.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 234414 3.39 MB/sec execute 260 sec latency 73.459 ms 2024-03-14T21:46:10.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 235557 3.39 MB/sec execute 261 sec latency 51.442 ms 2024-03-14T21:46:11.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 236550 3.39 MB/sec execute 262 sec latency 566.975 ms 2024-03-14T21:46:12.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 236550 3.38 MB/sec execute 263 sec latency 1567.099 ms 2024-03-14T21:46:13.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 236550 3.36 MB/sec execute 264 sec latency 2567.186 ms 2024-03-14T21:46:14.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 236550 3.35 MB/sec execute 265 sec latency 3567.267 ms 2024-03-14T21:46:15.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 236550 3.34 MB/sec execute 266 sec latency 4567.344 ms 2024-03-14T21:46:16.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 237416 3.35 MB/sec execute 267 sec latency 5001.247 ms 2024-03-14T21:46:17.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 238418 3.36 MB/sec execute 268 sec latency 48.863 ms 2024-03-14T21:46:18.676 INFO:tasks.workunit.client.0.smithi163.stdout: 1 239255 3.36 MB/sec execute 269 sec latency 244.035 ms 2024-03-14T21:46:19.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 241057 3.39 MB/sec execute 270 sec latency 50.557 ms 2024-03-14T21:46:20.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 242065 3.39 MB/sec execute 271 sec latency 46.932 ms 2024-03-14T21:46:21.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 243718 3.41 MB/sec execute 272 sec latency 52.836 ms 2024-03-14T21:46:22.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 244132 3.41 MB/sec execute 273 sec latency 812.389 ms 2024-03-14T21:46:23.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 244132 3.40 MB/sec execute 274 sec latency 1812.469 ms 2024-03-14T21:46:24.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 244132 3.39 MB/sec execute 275 sec latency 2812.543 ms 2024-03-14T21:46:25.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 244132 3.37 MB/sec execute 276 sec latency 3812.622 ms 2024-03-14T21:46:26.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 244132 3.36 MB/sec execute 277 sec latency 4812.713 ms 2024-03-14T21:46:27.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 245188 3.37 MB/sec execute 278 sec latency 5002.572 ms 2024-03-14T21:46:28.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 245957 3.37 MB/sec execute 279 sec latency 159.381 ms 2024-03-14T21:46:29.677 INFO:tasks.workunit.client.0.smithi163.stdout: 1 245967 3.36 MB/sec execute 280 sec latency 728.452 ms 2024-03-14T21:46:30.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 245974 3.35 MB/sec execute 281 sec latency 365.307 ms 2024-03-14T21:46:31.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 246009 3.33 MB/sec execute 282 sec latency 362.756 ms 2024-03-14T21:46:32.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 246018 3.32 MB/sec execute 283 sec latency 362.579 ms 2024-03-14T21:46:33.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 246038 3.31 MB/sec execute 284 sec latency 440.552 ms 2024-03-14T21:46:34.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 246274 3.31 MB/sec execute 285 sec latency 1114.944 ms 2024-03-14T21:46:35.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 246274 3.30 MB/sec execute 286 sec latency 1168.776 ms 2024-03-14T21:46:36.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 246274 3.29 MB/sec execute 287 sec latency 2168.807 ms 2024-03-14T21:46:37.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:46:37.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:46:37.475 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:46:37.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 247245 3.28 MB/sec execute 288 sec latency 2624.790 ms 2024-03-14T21:46:38.681 INFO:tasks.workunit.client.0.smithi163.stdout: 1 248736 3.31 MB/sec execute 289 sec latency 62.678 ms 2024-03-14T21:46:39.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 249969 3.32 MB/sec execute 290 sec latency 58.227 ms 2024-03-14T21:46:40.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 250689 3.31 MB/sec execute 291 sec latency 475.689 ms 2024-03-14T21:46:41.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 252197 3.34 MB/sec execute 292 sec latency 52.317 ms 2024-03-14T21:46:42.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 253398 3.35 MB/sec execute 293 sec latency 52.338 ms 2024-03-14T21:46:43.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 253795 3.34 MB/sec execute 294 sec latency 831.224 ms 2024-03-14T21:46:44.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 253795 3.33 MB/sec execute 295 sec latency 1831.273 ms 2024-03-14T21:46:45.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 253795 3.32 MB/sec execute 296 sec latency 2831.319 ms 2024-03-14T21:46:46.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 253795 3.31 MB/sec execute 297 sec latency 3831.366 ms 2024-03-14T21:46:47.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 253795 3.30 MB/sec execute 298 sec latency 4831.395 ms 2024-03-14T21:46:48.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 255386 3.32 MB/sec execute 299 sec latency 5003.291 ms 2024-03-14T21:46:49.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 256304 3.32 MB/sec execute 300 sec latency 50.311 ms 2024-03-14T21:46:50.682 INFO:tasks.workunit.client.0.smithi163.stdout: 1 257295 3.33 MB/sec execute 301 sec latency 170.393 ms 2024-03-14T21:46:51.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 258791 3.34 MB/sec execute 302 sec latency 202.670 ms 2024-03-14T21:46:52.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 259843 3.35 MB/sec execute 303 sec latency 50.407 ms 2024-03-14T21:46:53.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 260457 3.35 MB/sec execute 304 sec latency 592.341 ms 2024-03-14T21:46:54.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 260457 3.34 MB/sec execute 305 sec latency 1592.392 ms 2024-03-14T21:46:55.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 260457 3.33 MB/sec execute 306 sec latency 2592.440 ms 2024-03-14T21:46:56.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 260457 3.32 MB/sec execute 307 sec latency 3592.487 ms 2024-03-14T21:46:57.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 260457 3.31 MB/sec execute 308 sec latency 4592.530 ms 2024-03-14T21:46:58.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 261447 3.31 MB/sec execute 309 sec latency 5003.448 ms 2024-03-14T21:46:59.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 262899 3.33 MB/sec execute 310 sec latency 63.766 ms 2024-03-14T21:47:00.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 263676 3.33 MB/sec execute 311 sec latency 130.209 ms 2024-03-14T21:47:01.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 265491 3.35 MB/sec execute 312 sec latency 46.258 ms 2024-03-14T21:47:02.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 266736 3.36 MB/sec execute 313 sec latency 47.292 ms 2024-03-14T21:47:03.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 268407 3.38 MB/sec execute 314 sec latency 54.629 ms 2024-03-14T21:47:04.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 268528 3.37 MB/sec execute 315 sec latency 839.464 ms 2024-03-14T21:47:05.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 268528 3.36 MB/sec execute 316 sec latency 1839.500 ms 2024-03-14T21:47:06.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 268528 3.35 MB/sec execute 317 sec latency 2839.532 ms 2024-03-14T21:47:07.600 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:47:07.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:47:07.606 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:47:07.683 INFO:tasks.workunit.client.0.smithi163.stdout: 1 268528 3.34 MB/sec execute 318 sec latency 3839.568 ms 2024-03-14T21:47:08.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 268528 3.32 MB/sec execute 319 sec latency 4839.619 ms 2024-03-14T21:47:09.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 269891 3.35 MB/sec execute 320 sec latency 5001.808 ms 2024-03-14T21:47:10.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270503 3.34 MB/sec execute 321 sec latency 181.943 ms 2024-03-14T21:47:11.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270515 3.33 MB/sec execute 322 sec latency 724.276 ms 2024-03-14T21:47:12.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270516 3.32 MB/sec execute 323 sec latency 1085.031 ms 2024-03-14T21:47:13.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270526 3.31 MB/sec execute 324 sec latency 638.700 ms 2024-03-14T21:47:14.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270539 3.30 MB/sec execute 325 sec latency 729.892 ms 2024-03-14T21:47:15.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270539 3.29 MB/sec execute 326 sec latency 1178.971 ms 2024-03-14T21:47:16.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270539 3.28 MB/sec execute 327 sec latency 2179.037 ms 2024-03-14T21:47:17.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270539 3.27 MB/sec execute 328 sec latency 3179.076 ms 2024-03-14T21:47:18.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 270539 3.26 MB/sec execute 329 sec latency 4179.107 ms 2024-03-14T21:47:19.684 INFO:tasks.workunit.client.0.smithi163.stdout: 1 271240 3.26 MB/sec execute 330 sec latency 4590.898 ms 2024-03-14T21:47:20.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 272979 3.28 MB/sec execute 331 sec latency 53.281 ms 2024-03-14T21:47:21.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 273961 3.29 MB/sec execute 332 sec latency 59.337 ms 2024-03-14T21:47:22.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 274369 3.28 MB/sec execute 333 sec latency 555.387 ms 2024-03-14T21:47:23.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 276171 3.30 MB/sec execute 334 sec latency 49.728 ms 2024-03-14T21:47:24.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 277426 3.32 MB/sec execute 335 sec latency 55.954 ms 2024-03-14T21:47:25.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 279129 3.33 MB/sec execute 336 sec latency 84.389 ms 2024-03-14T21:47:26.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 280725 3.35 MB/sec execute 337 sec latency 41.096 ms 2024-03-14T21:47:27.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 280975 3.34 MB/sec execute 338 sec latency 801.610 ms 2024-03-14T21:47:28.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 280975 3.33 MB/sec execute 339 sec latency 1801.645 ms 2024-03-14T21:47:29.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 280975 3.32 MB/sec execute 340 sec latency 2801.674 ms 2024-03-14T21:47:30.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 282542 3.33 MB/sec execute 341 sec latency 2963.250 ms 2024-03-14T21:47:31.685 INFO:tasks.workunit.client.0.smithi163.stdout: 1 284078 3.36 MB/sec execute 342 sec latency 50.422 ms 2024-03-14T21:47:32.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 284461 3.35 MB/sec execute 343 sec latency 336.092 ms 2024-03-14T21:47:33.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 285443 3.35 MB/sec execute 344 sec latency 583.111 ms 2024-03-14T21:47:34.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 287558 3.38 MB/sec execute 345 sec latency 43.283 ms 2024-03-14T21:47:35.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 287838 3.37 MB/sec execute 346 sec latency 608.096 ms 2024-03-14T21:47:36.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 287838 3.36 MB/sec execute 347 sec latency 1608.145 ms 2024-03-14T21:47:37.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 287838 3.36 MB/sec execute 348 sec latency 2608.178 ms 2024-03-14T21:47:37.735 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:47:37.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:47:37.740 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:47:38.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 287838 3.35 MB/sec execute 349 sec latency 3608.216 ms 2024-03-14T21:47:39.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 287838 3.34 MB/sec execute 350 sec latency 4608.253 ms 2024-03-14T21:47:40.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 288322 3.33 MB/sec execute 351 sec latency 5029.398 ms 2024-03-14T21:47:41.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 290342 3.35 MB/sec execute 352 sec latency 46.830 ms 2024-03-14T21:47:42.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 291552 3.36 MB/sec execute 353 sec latency 64.108 ms 2024-03-14T21:47:43.686 INFO:tasks.workunit.client.0.smithi163.stdout: 1 292363 3.37 MB/sec execute 354 sec latency 448.520 ms 2024-03-14T21:47:44.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 294482 3.39 MB/sec execute 355 sec latency 55.888 ms 2024-03-14T21:47:45.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 295338 3.39 MB/sec execute 356 sec latency 59.457 ms 2024-03-14T21:47:46.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 295497 3.38 MB/sec execute 357 sec latency 860.286 ms 2024-03-14T21:47:47.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 295497 3.37 MB/sec execute 358 sec latency 1860.321 ms 2024-03-14T21:47:48.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 295497 3.36 MB/sec execute 359 sec latency 2860.365 ms 2024-03-14T21:47:49.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 295497 3.35 MB/sec execute 360 sec latency 3860.408 ms 2024-03-14T21:47:50.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 295497 3.35 MB/sec execute 361 sec latency 4860.436 ms 2024-03-14T21:47:51.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 297198 3.36 MB/sec execute 362 sec latency 5003.258 ms 2024-03-14T21:47:52.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298332 3.37 MB/sec execute 363 sec latency 198.669 ms 2024-03-14T21:47:53.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298345 3.37 MB/sec execute 364 sec latency 365.353 ms 2024-03-14T21:47:54.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298349 3.36 MB/sec execute 365 sec latency 724.487 ms 2024-03-14T21:47:55.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298349 3.35 MB/sec execute 366 sec latency 1012.785 ms 2024-03-14T21:47:56.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298356 3.34 MB/sec execute 367 sec latency 1451.138 ms 2024-03-14T21:47:57.687 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298361 3.33 MB/sec execute 368 sec latency 733.322 ms 2024-03-14T21:47:58.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298416 3.32 MB/sec execute 369 sec latency 726.828 ms 2024-03-14T21:47:59.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298416 3.31 MB/sec execute 370 sec latency 1104.232 ms 2024-03-14T21:48:00.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298416 3.30 MB/sec execute 371 sec latency 2104.308 ms 2024-03-14T21:48:01.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 298774 3.29 MB/sec execute 372 sec latency 2579.317 ms 2024-03-14T21:48:02.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 300382 3.30 MB/sec execute 373 sec latency 50.806 ms 2024-03-14T21:48:03.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 301834 3.33 MB/sec execute 374 sec latency 48.956 ms 2024-03-14T21:48:04.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 302406 3.32 MB/sec execute 375 sec latency 118.533 ms 2024-03-14T21:48:05.688 INFO:tasks.workunit.client.0.smithi163.stdout: 1 303957 3.33 MB/sec execute 376 sec latency 55.736 ms 2024-03-14T21:48:06.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 305452 3.35 MB/sec execute 377 sec latency 45.798 ms 2024-03-14T21:48:07.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 305949 3.34 MB/sec execute 378 sec latency 392.964 ms 2024-03-14T21:48:08.001 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:48:08.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:48:08.006 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:48:08.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 305949 3.33 MB/sec execute 379 sec latency 1393.022 ms 2024-03-14T21:48:09.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 305949 3.33 MB/sec execute 380 sec latency 2393.087 ms 2024-03-14T21:48:10.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 305949 3.32 MB/sec execute 381 sec latency 3393.159 ms 2024-03-14T21:48:11.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 305949 3.31 MB/sec execute 382 sec latency 4393.217 ms 2024-03-14T21:48:12.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 306500 3.31 MB/sec execute 383 sec latency 5037.998 ms 2024-03-14T21:48:13.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 308231 3.32 MB/sec execute 384 sec latency 63.196 ms 2024-03-14T21:48:14.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 309073 3.33 MB/sec execute 385 sec latency 55.221 ms 2024-03-14T21:48:15.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 310305 3.34 MB/sec execute 386 sec latency 83.629 ms 2024-03-14T21:48:16.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 312293 3.36 MB/sec execute 387 sec latency 49.041 ms 2024-03-14T21:48:17.689 INFO:tasks.workunit.client.0.smithi163.stdout: 1 313135 3.36 MB/sec execute 388 sec latency 62.569 ms 2024-03-14T21:48:18.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 313981 3.36 MB/sec execute 389 sec latency 607.195 ms 2024-03-14T21:48:19.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 313981 3.35 MB/sec execute 390 sec latency 1607.283 ms 2024-03-14T21:48:20.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 313981 3.34 MB/sec execute 391 sec latency 2607.379 ms 2024-03-14T21:48:21.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 313981 3.33 MB/sec execute 392 sec latency 3607.455 ms 2024-03-14T21:48:22.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 313981 3.33 MB/sec execute 393 sec latency 4607.542 ms 2024-03-14T21:48:23.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 315056 3.33 MB/sec execute 394 sec latency 5003.381 ms 2024-03-14T21:48:24.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 316150 3.34 MB/sec execute 395 sec latency 42.660 ms 2024-03-14T21:48:25.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 317394 3.35 MB/sec execute 396 sec latency 59.602 ms 2024-03-14T21:48:26.690 INFO:tasks.workunit.client.0.smithi163.stdout: 1 319458 3.37 MB/sec execute 397 sec latency 45.557 ms 2024-03-14T21:48:27.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 320236 3.37 MB/sec execute 398 sec latency 57.413 ms 2024-03-14T21:48:28.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 322166 3.39 MB/sec execute 399 sec latency 52.518 ms 2024-03-14T21:48:29.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 322441 3.38 MB/sec execute 400 sec latency 867.548 ms 2024-03-14T21:48:30.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 322441 3.37 MB/sec execute 401 sec latency 1867.631 ms 2024-03-14T21:48:31.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 322441 3.37 MB/sec execute 402 sec latency 2867.705 ms 2024-03-14T21:48:32.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 322441 3.36 MB/sec execute 403 sec latency 3867.781 ms 2024-03-14T21:48:33.691 INFO:tasks.workunit.client.0.smithi163.stdout: 1 322441 3.35 MB/sec execute 404 sec latency 4867.885 ms 2024-03-14T21:48:34.692 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323202 3.36 MB/sec execute 405 sec latency 5014.355 ms 2024-03-14T21:48:35.692 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323214 3.35 MB/sec execute 406 sec latency 368.332 ms 2024-03-14T21:48:36.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323225 3.34 MB/sec execute 407 sec latency 367.668 ms 2024-03-14T21:48:37.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323230 3.33 MB/sec execute 408 sec latency 367.557 ms 2024-03-14T21:48:38.290 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:48:38.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:48:38.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:48:38.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323237 3.32 MB/sec execute 409 sec latency 726.218 ms 2024-03-14T21:48:39.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323253 3.31 MB/sec execute 410 sec latency 370.971 ms 2024-03-14T21:48:40.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323257 3.31 MB/sec execute 411 sec latency 795.640 ms 2024-03-14T21:48:41.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323257 3.30 MB/sec execute 412 sec latency 1795.700 ms 2024-03-14T21:48:42.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323257 3.29 MB/sec execute 413 sec latency 2795.731 ms 2024-03-14T21:48:43.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323257 3.28 MB/sec execute 414 sec latency 3795.760 ms 2024-03-14T21:48:44.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 323784 3.28 MB/sec execute 415 sec latency 4156.872 ms 2024-03-14T21:48:45.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 325396 3.29 MB/sec execute 416 sec latency 46.286 ms 2024-03-14T21:48:46.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 326600 3.30 MB/sec execute 417 sec latency 118.646 ms 2024-03-14T21:48:47.741 INFO:tasks.workunit.client.0.smithi163.stdout: 1 327405 3.30 MB/sec execute 418 sec latency 60.663 ms 2024-03-14T21:48:48.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 329278 3.32 MB/sec execute 419 sec latency 45.377 ms 2024-03-14T21:48:49.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 330562 3.33 MB/sec execute 420 sec latency 33.734 ms 2024-03-14T21:48:50.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 330618 3.32 MB/sec execute 421 sec latency 951.039 ms 2024-03-14T21:48:51.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 330618 3.31 MB/sec execute 422 sec latency 1951.148 ms 2024-03-14T21:48:52.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 330618 3.30 MB/sec execute 423 sec latency 2951.267 ms 2024-03-14T21:48:53.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 330618 3.30 MB/sec execute 424 sec latency 3951.373 ms 2024-03-14T21:48:54.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 330618 3.29 MB/sec execute 425 sec latency 4951.474 ms 2024-03-14T21:48:55.742 INFO:tasks.workunit.client.0.smithi163.stdout: 1 331731 3.29 MB/sec execute 426 sec latency 5038.682 ms 2024-03-14T21:48:56.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 332364 3.29 MB/sec execute 427 sec latency 649.425 ms 2024-03-14T21:48:57.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 334022 3.31 MB/sec execute 428 sec latency 48.096 ms 2024-03-14T21:48:58.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 335782 3.32 MB/sec execute 429 sec latency 56.293 ms 2024-03-14T21:48:59.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 337402 3.34 MB/sec execute 430 sec latency 51.298 ms 2024-03-14T21:49:00.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 337709 3.33 MB/sec execute 431 sec latency 703.872 ms 2024-03-14T21:49:01.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 337709 3.32 MB/sec execute 432 sec latency 1703.971 ms 2024-03-14T21:49:02.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 337709 3.32 MB/sec execute 433 sec latency 2704.079 ms 2024-03-14T21:49:03.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 337709 3.31 MB/sec execute 434 sec latency 3704.182 ms 2024-03-14T21:49:04.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 337709 3.30 MB/sec execute 435 sec latency 4704.286 ms 2024-03-14T21:49:05.743 INFO:tasks.workunit.client.0.smithi163.stdout: 1 338673 3.30 MB/sec execute 436 sec latency 5030.416 ms 2024-03-14T21:49:06.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 340627 3.32 MB/sec execute 437 sec latency 132.904 ms 2024-03-14T21:49:07.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 341508 3.32 MB/sec execute 438 sec latency 54.388 ms 2024-03-14T21:49:08.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:49:08.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:49:08.601 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:49:08.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 343714 3.34 MB/sec execute 439 sec latency 45.944 ms 2024-03-14T21:49:09.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 344880 3.35 MB/sec execute 440 sec latency 54.539 ms 2024-03-14T21:49:10.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 345762 3.35 MB/sec execute 441 sec latency 438.424 ms 2024-03-14T21:49:11.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 345762 3.35 MB/sec execute 442 sec latency 1438.508 ms 2024-03-14T21:49:12.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 345762 3.34 MB/sec execute 443 sec latency 2438.585 ms 2024-03-14T21:49:13.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 345762 3.33 MB/sec execute 444 sec latency 3438.673 ms 2024-03-14T21:49:14.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 345762 3.32 MB/sec execute 445 sec latency 4438.791 ms 2024-03-14T21:49:15.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 346545 3.32 MB/sec execute 446 sec latency 5012.377 ms 2024-03-14T21:49:16.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 347881 3.34 MB/sec execute 447 sec latency 290.506 ms 2024-03-14T21:49:17.745 INFO:tasks.workunit.client.0.smithi163.stdout: 1 347889 3.33 MB/sec execute 448 sec latency 366.564 ms 2024-03-14T21:49:18.744 INFO:tasks.workunit.client.0.smithi163.stdout: 1 347897 3.32 MB/sec execute 449 sec latency 466.897 ms 2024-03-14T21:49:19.752 INFO:tasks.workunit.client.0.smithi163.stdout: 1 347897 3.32 MB/sec execute 450 sec latency 1474.314 ms 2024-03-14T21:49:20.752 INFO:tasks.workunit.client.0.smithi163.stdout: 1 347901 3.31 MB/sec execute 451 sec latency 2180.759 ms 2024-03-14T21:49:21.752 INFO:tasks.workunit.client.0.smithi163.stdout: 1 347917 3.30 MB/sec execute 452 sec latency 373.812 ms 2024-03-14T21:49:22.752 INFO:tasks.workunit.client.0.smithi163.stdout: 1 348287 3.30 MB/sec execute 453 sec latency 359.740 ms 2024-03-14T21:49:23.752 INFO:tasks.workunit.client.0.smithi163.stdout: 1 348287 3.29 MB/sec execute 454 sec latency 1064.284 ms 2024-03-14T21:49:24.752 INFO:tasks.workunit.client.0.smithi163.stdout: 1 348287 3.28 MB/sec execute 455 sec latency 2064.360 ms 2024-03-14T21:49:25.752 INFO:tasks.workunit.client.0.smithi163.stdout: 1 348287 3.27 MB/sec execute 456 sec latency 3064.436 ms 2024-03-14T21:49:26.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 348718 3.27 MB/sec execute 457 sec latency 3529.967 ms 2024-03-14T21:49:27.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 350903 3.29 MB/sec execute 458 sec latency 42.242 ms 2024-03-14T21:49:28.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 351604 3.29 MB/sec execute 459 sec latency 406.572 ms 2024-03-14T21:49:29.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 353137 3.30 MB/sec execute 460 sec latency 54.755 ms 2024-03-14T21:49:30.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 355012 3.32 MB/sec execute 461 sec latency 59.693 ms 2024-03-14T21:49:31.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 355501 3.31 MB/sec execute 462 sec latency 469.760 ms 2024-03-14T21:49:32.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 355501 3.31 MB/sec execute 463 sec latency 1469.885 ms 2024-03-14T21:49:33.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 355501 3.30 MB/sec execute 464 sec latency 2469.965 ms 2024-03-14T21:49:34.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 355501 3.29 MB/sec execute 465 sec latency 3470.059 ms 2024-03-14T21:49:35.753 INFO:tasks.workunit.client.0.smithi163.stdout: 1 355501 3.29 MB/sec execute 466 sec latency 4470.135 ms 2024-03-14T21:49:36.754 INFO:tasks.workunit.client.0.smithi163.stdout: 1 356148 3.29 MB/sec execute 467 sec latency 5017.944 ms 2024-03-14T21:49:37.754 INFO:tasks.workunit.client.0.smithi163.stdout: 1 358051 3.30 MB/sec execute 468 sec latency 72.967 ms 2024-03-14T21:49:38.754 INFO:tasks.workunit.client.0.smithi163.stdout: 1 358926 3.30 MB/sec execute 469 sec latency 50.782 ms 2024-03-14T21:49:38.909 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:49:38.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:49:38.914 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:49:39.754 INFO:tasks.workunit.client.0.smithi163.stdout: 1 360585 3.31 MB/sec execute 470 sec latency 81.835 ms 2024-03-14T21:49:40.754 INFO:tasks.workunit.client.0.smithi163.stdout: 1 362087 3.33 MB/sec execute 471 sec latency 50.799 ms 2024-03-14T21:49:41.754 INFO:tasks.workunit.client.0.smithi163.stdout: 1 363075 3.33 MB/sec execute 472 sec latency 54.965 ms 2024-03-14T21:49:42.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 363882 3.33 MB/sec execute 473 sec latency 702.284 ms 2024-03-14T21:49:43.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 363882 3.32 MB/sec execute 474 sec latency 1702.371 ms 2024-03-14T21:49:44.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 363882 3.31 MB/sec execute 475 sec latency 2702.490 ms 2024-03-14T21:49:45.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 363882 3.31 MB/sec execute 476 sec latency 3702.556 ms 2024-03-14T21:49:46.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 363882 3.30 MB/sec execute 477 sec latency 4702.660 ms 2024-03-14T21:49:47.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 365086 3.31 MB/sec execute 478 sec latency 5029.533 ms 2024-03-14T21:49:48.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 365915 3.32 MB/sec execute 479 sec latency 36.578 ms 2024-03-14T21:49:49.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 367008 3.32 MB/sec execute 480 sec latency 139.081 ms 2024-03-14T21:49:50.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 368962 3.34 MB/sec execute 481 sec latency 45.375 ms 2024-03-14T21:49:51.755 INFO:tasks.workunit.client.0.smithi163.stdout: 1 369767 3.34 MB/sec execute 482 sec latency 50.645 ms 2024-03-14T21:49:52.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 370710 3.34 MB/sec execute 483 sec latency 463.048 ms 2024-03-14T21:49:53.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 370710 3.33 MB/sec execute 484 sec latency 1463.146 ms 2024-03-14T21:49:54.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 370710 3.32 MB/sec execute 485 sec latency 2463.231 ms 2024-03-14T21:49:55.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 370710 3.32 MB/sec execute 486 sec latency 3463.308 ms 2024-03-14T21:49:56.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 370710 3.31 MB/sec execute 487 sec latency 4463.398 ms 2024-03-14T21:49:57.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 371382 3.31 MB/sec execute 488 sec latency 5007.766 ms 2024-03-14T21:49:58.757 INFO:tasks.workunit.client.0.smithi163.stdout: 1 372851 3.33 MB/sec execute 489 sec latency 46.859 ms 2024-03-14T21:49:59.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373080 3.32 MB/sec execute 490 sec latency 366.274 ms 2024-03-14T21:50:00.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373098 3.31 MB/sec execute 491 sec latency 365.514 ms 2024-03-14T21:50:01.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373108 3.31 MB/sec execute 492 sec latency 725.509 ms 2024-03-14T21:50:02.756 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373109 3.30 MB/sec execute 493 sec latency 1087.235 ms 2024-03-14T21:50:03.784 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373118 3.29 MB/sec execute 494 sec latency 365.742 ms 2024-03-14T21:50:04.784 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373154 3.29 MB/sec execute 495 sec latency 362.351 ms 2024-03-14T21:50:05.784 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373164 3.28 MB/sec execute 496 sec latency 662.117 ms 2024-03-14T21:50:06.784 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373164 3.27 MB/sec execute 497 sec latency 1662.198 ms 2024-03-14T21:50:07.784 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373164 3.27 MB/sec execute 498 sec latency 2662.252 ms 2024-03-14T21:50:08.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 373950 3.27 MB/sec execute 499 sec latency 3013.269 ms 2024-03-14T21:50:09.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:50:09.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:50:09.067 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:50:09.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 375605 3.28 MB/sec execute 500 sec latency 46.565 ms 2024-03-14T21:50:10.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 376269 3.28 MB/sec execute 501 sec latency 459.735 ms 2024-03-14T21:50:11.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 376964 3.28 MB/sec execute 502 sec latency 56.512 ms 2024-03-14T21:50:12.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 378906 3.29 MB/sec execute 503 sec latency 57.597 ms 2024-03-14T21:50:13.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 379782 3.30 MB/sec execute 504 sec latency 502.949 ms 2024-03-14T21:50:14.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 379782 3.30 MB/sec execute 505 sec latency 1503.036 ms 2024-03-14T21:50:15.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 379782 3.29 MB/sec execute 506 sec latency 2503.121 ms 2024-03-14T21:50:16.785 INFO:tasks.workunit.client.0.smithi163.stdout: 1 379782 3.28 MB/sec execute 507 sec latency 3503.208 ms 2024-03-14T21:50:17.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 379782 3.28 MB/sec execute 508 sec latency 4503.289 ms 2024-03-14T21:50:18.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 380018 3.27 MB/sec execute 509 sec latency 5015.357 ms 2024-03-14T21:50:19.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 381172 3.27 MB/sec execute 510 sec latency 62.456 ms 2024-03-14T21:50:20.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 382491 3.28 MB/sec execute 511 sec latency 103.918 ms 2024-03-14T21:50:21.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 383781 3.29 MB/sec execute 512 sec latency 22.629 ms 2024-03-14T21:50:22.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 385475 3.30 MB/sec execute 513 sec latency 60.378 ms 2024-03-14T21:50:23.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 386945 3.31 MB/sec execute 514 sec latency 44.921 ms 2024-03-14T21:50:24.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 387093 3.31 MB/sec execute 515 sec latency 749.665 ms 2024-03-14T21:50:25.786 INFO:tasks.workunit.client.0.smithi163.stdout: 1 387093 3.30 MB/sec execute 516 sec latency 1749.781 ms 2024-03-14T21:50:26.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 387093 3.29 MB/sec execute 517 sec latency 2749.901 ms 2024-03-14T21:50:27.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 387093 3.29 MB/sec execute 518 sec latency 3750.011 ms 2024-03-14T21:50:28.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 387093 3.28 MB/sec execute 519 sec latency 4750.122 ms 2024-03-14T21:50:29.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 387603 3.28 MB/sec execute 520 sec latency 5022.105 ms 2024-03-14T21:50:30.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 388767 3.28 MB/sec execute 521 sec latency 377.177 ms 2024-03-14T21:50:31.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 390467 3.30 MB/sec execute 522 sec latency 47.765 ms 2024-03-14T21:50:32.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 391209 3.29 MB/sec execute 523 sec latency 53.112 ms 2024-03-14T21:50:33.787 INFO:tasks.workunit.client.0.smithi163.stdout: 1 393142 3.31 MB/sec execute 524 sec latency 53.829 ms 2024-03-14T21:50:34.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 394008 3.32 MB/sec execute 525 sec latency 510.045 ms 2024-03-14T21:50:35.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 394008 3.31 MB/sec execute 526 sec latency 1510.151 ms 2024-03-14T21:50:36.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 394008 3.30 MB/sec execute 527 sec latency 2510.253 ms 2024-03-14T21:50:37.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 394008 3.30 MB/sec execute 528 sec latency 3510.360 ms 2024-03-14T21:50:38.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 394008 3.29 MB/sec execute 529 sec latency 4510.444 ms 2024-03-14T21:50:39.197 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:50:39.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:50:39.203 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:50:39.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 394443 3.29 MB/sec execute 530 sec latency 5008.869 ms 2024-03-14T21:50:40.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 395907 3.29 MB/sec execute 531 sec latency 79.074 ms 2024-03-14T21:50:41.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396163 3.29 MB/sec execute 532 sec latency 368.977 ms 2024-03-14T21:50:42.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396176 3.28 MB/sec execute 533 sec latency 362.564 ms 2024-03-14T21:50:43.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396181 3.27 MB/sec execute 534 sec latency 724.227 ms 2024-03-14T21:50:44.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396185 3.27 MB/sec execute 535 sec latency 584.409 ms 2024-03-14T21:50:45.788 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396197 3.26 MB/sec execute 536 sec latency 1089.074 ms 2024-03-14T21:50:46.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396202 3.26 MB/sec execute 537 sec latency 727.532 ms 2024-03-14T21:50:47.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396202 3.25 MB/sec execute 538 sec latency 1396.871 ms 2024-03-14T21:50:48.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396202 3.24 MB/sec execute 539 sec latency 2396.949 ms 2024-03-14T21:50:49.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 396202 3.24 MB/sec execute 540 sec latency 3397.052 ms 2024-03-14T21:50:50.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 397521 3.25 MB/sec execute 541 sec latency 3637.223 ms 2024-03-14T21:50:51.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 398212 3.25 MB/sec execute 542 sec latency 56.967 ms 2024-03-14T21:50:52.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 399687 3.25 MB/sec execute 543 sec latency 105.374 ms 2024-03-14T21:50:53.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 401162 3.27 MB/sec execute 544 sec latency 44.696 ms 2024-03-14T21:50:54.789 INFO:tasks.workunit.client.0.smithi163.stdout: 1 402037 3.27 MB/sec execute 545 sec latency 59.677 ms 2024-03-14T21:50:55.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 403213 3.27 MB/sec execute 546 sec latency 521.134 ms 2024-03-14T21:50:56.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 403213 3.27 MB/sec execute 547 sec latency 1521.253 ms 2024-03-14T21:50:57.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 403213 3.26 MB/sec execute 548 sec latency 2521.364 ms 2024-03-14T21:50:58.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 403213 3.25 MB/sec execute 549 sec latency 3521.494 ms 2024-03-14T21:50:59.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 403213 3.25 MB/sec execute 550 sec latency 4521.577 ms 2024-03-14T21:51:00.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 403837 3.25 MB/sec execute 551 sec latency 5023.551 ms 2024-03-14T21:51:01.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 404982 3.26 MB/sec execute 552 sec latency 27.820 ms 2024-03-14T21:51:02.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 405284 3.25 MB/sec execute 553 sec latency 412.879 ms 2024-03-14T21:51:03.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 406818 3.26 MB/sec execute 554 sec latency 306.194 ms 2024-03-14T21:51:04.790 INFO:tasks.workunit.client.0.smithi163.stdout: 1 408290 3.27 MB/sec execute 555 sec latency 49.549 ms 2024-03-14T21:51:05.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 409073 3.27 MB/sec execute 556 sec latency 67.671 ms 2024-03-14T21:51:06.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 409542 3.27 MB/sec execute 557 sec latency 769.956 ms 2024-03-14T21:51:07.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 409542 3.27 MB/sec execute 558 sec latency 1770.118 ms 2024-03-14T21:51:08.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 409542 3.26 MB/sec execute 559 sec latency 2770.227 ms 2024-03-14T21:51:09.310 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:51:09.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:51:09.316 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:51:09.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 409542 3.25 MB/sec execute 560 sec latency 3770.323 ms 2024-03-14T21:51:10.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 409542 3.25 MB/sec execute 561 sec latency 4770.433 ms 2024-03-14T21:51:11.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 410715 3.25 MB/sec execute 562 sec latency 5026.130 ms 2024-03-14T21:51:12.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 411948 3.26 MB/sec execute 563 sec latency 21.355 ms 2024-03-14T21:51:13.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 413131 3.27 MB/sec execute 564 sec latency 101.492 ms 2024-03-14T21:51:14.791 INFO:tasks.workunit.client.0.smithi163.stdout: 1 414922 3.28 MB/sec execute 565 sec latency 62.342 ms 2024-03-14T21:51:15.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 415841 3.28 MB/sec execute 566 sec latency 53.819 ms 2024-03-14T21:51:16.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 416546 3.28 MB/sec execute 567 sec latency 532.937 ms 2024-03-14T21:51:17.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 416546 3.28 MB/sec execute 568 sec latency 1533.029 ms 2024-03-14T21:51:18.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 416546 3.27 MB/sec execute 569 sec latency 2533.129 ms 2024-03-14T21:51:19.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 416546 3.26 MB/sec execute 570 sec latency 3533.237 ms 2024-03-14T21:51:20.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 416546 3.26 MB/sec execute 571 sec latency 4533.335 ms 2024-03-14T21:51:21.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 417495 3.26 MB/sec execute 572 sec latency 5007.197 ms 2024-03-14T21:51:22.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 418990 3.27 MB/sec execute 573 sec latency 46.917 ms 2024-03-14T21:51:23.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419110 3.27 MB/sec execute 574 sec latency 485.337 ms 2024-03-14T21:51:24.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419203 3.26 MB/sec execute 575 sec latency 734.183 ms 2024-03-14T21:51:25.792 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419245 3.26 MB/sec execute 576 sec latency 717.521 ms 2024-03-14T21:51:26.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419250 3.25 MB/sec execute 577 sec latency 927.533 ms 2024-03-14T21:51:27.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419254 3.24 MB/sec execute 578 sec latency 1086.568 ms 2024-03-14T21:51:28.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419277 3.24 MB/sec execute 579 sec latency 362.318 ms 2024-03-14T21:51:29.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419277 3.23 MB/sec execute 580 sec latency 1150.012 ms 2024-03-14T21:51:30.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419277 3.23 MB/sec execute 581 sec latency 2150.100 ms 2024-03-14T21:51:31.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419277 3.22 MB/sec execute 582 sec latency 3150.177 ms 2024-03-14T21:51:32.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 419573 3.22 MB/sec execute 583 sec latency 3672.392 ms 2024-03-14T21:51:33.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 421476 3.23 MB/sec execute 584 sec latency 59.322 ms 2024-03-14T21:51:34.793 INFO:tasks.workunit.client.0.smithi163.stdout: 1 422579 3.24 MB/sec execute 585 sec latency 24.896 ms 2024-03-14T21:51:35.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 423827 3.24 MB/sec execute 586 sec latency 72.417 ms 2024-03-14T21:51:36.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 425807 3.26 MB/sec execute 587 sec latency 52.952 ms 2024-03-14T21:51:37.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 426597 3.25 MB/sec execute 588 sec latency 51.889 ms 2024-03-14T21:51:38.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 427588 3.26 MB/sec execute 589 sec latency 513.426 ms 2024-03-14T21:51:39.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:51:39.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:51:39.448 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:51:39.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 427588 3.25 MB/sec execute 590 sec latency 1513.523 ms 2024-03-14T21:51:40.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 427588 3.25 MB/sec execute 591 sec latency 2513.603 ms 2024-03-14T21:51:41.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 427588 3.24 MB/sec execute 592 sec latency 3513.690 ms 2024-03-14T21:51:42.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 427588 3.24 MB/sec execute 593 sec latency 4513.779 ms 2024-03-14T21:51:43.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 428256 3.23 MB/sec execute 594 sec latency 5027.433 ms 2024-03-14T21:51:44.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 429232 3.24 MB/sec execute 595 sec latency 368.946 ms 2024-03-14T21:51:45.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 430094 3.24 MB/sec execute 596 sec latency 64.336 ms 2024-03-14T21:51:46.794 INFO:tasks.workunit.client.0.smithi163.stdout: 1 431663 3.25 MB/sec execute 597 sec latency 54.407 ms 2024-03-14T21:51:47.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 433167 3.26 MB/sec execute 598 sec latency 41.934 ms 2024-03-14T21:51:48.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 434237 3.27 MB/sec execute 599 sec latency 60.881 ms 2024-03-14T21:51:49.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 cleanup 600 sec 2024-03-14T21:51:50.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 cleanup 601 sec 2024-03-14T21:51:51.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 cleanup 602 sec 2024-03-14T21:51:52.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 cleanup 603 sec 2024-03-14T21:51:53.795 INFO:tasks.workunit.client.0.smithi163.stdout: 1 cleanup 604 sec 2024-03-14T21:51:54.176 INFO:tasks.workunit.client.0.smithi163.stdout: 0 cleanup 605 sec 2024-03-14T21:51:54.176 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Operation Count AvgLat MaxLat 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: ---------------------------------------- 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: NTCreateX 62970 3.375 5362.601 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Close 46225 0.024 362.327 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Rename 2663 6.774 1114.928 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Unlink 12740 3.519 5004.298 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Qpathinfo 57088 2.375 5029.524 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Qfileinfo 9938 0.005 0.044 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Qfsinfo 10449 1.242 3672.376 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Sfileinfo 5134 0.955 366.875 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: Find 22028 0.387 734.172 2024-03-14T21:51:54.177 INFO:tasks.workunit.client.0.smithi163.stdout: WriteX 31062 0.280 362.827 2024-03-14T21:51:54.178 INFO:tasks.workunit.client.0.smithi163.stdout: ReadX 98504 0.013 32.811 2024-03-14T21:51:54.178 INFO:tasks.workunit.client.0.smithi163.stdout: LockX 204 0.745 5.798 2024-03-14T21:51:54.178 INFO:tasks.workunit.client.0.smithi163.stdout: UnlockX 204 0.662 4.036 2024-03-14T21:51:54.178 INFO:tasks.workunit.client.0.smithi163.stdout: Flush 4416 34.692 5038.672 2024-03-14T21:51:54.178 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:51:54.178 INFO:tasks.workunit.client.0.smithi163.stdout:Throughput 3.26606 MB/sec 1 clients 1 procs max_latency=5362.619 ms 2024-03-14T21:51:54.181 INFO:tasks.workunit.client.0.smithi163.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-14T21:51:54.181 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T21:51:54.181 INFO:tasks.workunit.client.0.smithi163.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-14T21:51:54.182 INFO:tasks.workunit.client.0.smithi163.stdout:9 of 10 processes prepared for launch 0 sec 2024-03-14T21:51:54.283 INFO:tasks.workunit.client.0.smithi163.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-14T21:51:54.283 INFO:tasks.workunit.client.0.smithi163.stdout:releasing clients 2024-03-14T21:51:55.283 INFO:tasks.workunit.client.0.smithi163.stdout: 10 237 90.91 MB/sec warmup 1 sec latency 254.232 ms 2024-03-14T21:51:56.283 INFO:tasks.workunit.client.0.smithi163.stdout: 10 647 103.31 MB/sec warmup 2 sec latency 92.686 ms 2024-03-14T21:51:57.283 INFO:tasks.workunit.client.0.smithi163.stdout: 10 913 72.71 MB/sec warmup 3 sec latency 297.973 ms 2024-03-14T21:51:58.283 INFO:tasks.workunit.client.0.smithi163.stdout: 10 1638 64.51 MB/sec warmup 4 sec latency 165.720 ms 2024-03-14T21:51:59.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2389 57.14 MB/sec warmup 5 sec latency 109.982 ms 2024-03-14T21:52:00.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2389 47.61 MB/sec warmup 6 sec latency 1032.735 ms 2024-03-14T21:52:01.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2389 40.81 MB/sec warmup 7 sec latency 2032.813 ms 2024-03-14T21:52:02.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2389 35.71 MB/sec warmup 8 sec latency 3032.939 ms 2024-03-14T21:52:03.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2389 31.74 MB/sec warmup 9 sec latency 4033.017 ms 2024-03-14T21:52:04.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2392 28.57 MB/sec warmup 10 sec latency 5026.776 ms 2024-03-14T21:52:05.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 2904 29.95 MB/sec warmup 11 sec latency 5015.270 ms 2024-03-14T21:52:06.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 3215 29.67 MB/sec warmup 12 sec latency 1160.843 ms 2024-03-14T21:52:07.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 3616 29.50 MB/sec warmup 13 sec latency 1246.216 ms 2024-03-14T21:52:08.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 3820 27.64 MB/sec warmup 14 sec latency 98.027 ms 2024-03-14T21:52:09.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4079 26.12 MB/sec warmup 15 sec latency 131.774 ms 2024-03-14T21:52:09.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:52:09.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:52:09.590 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:52:10.284 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4334 25.11 MB/sec warmup 16 sec latency 276.810 ms 2024-03-14T21:52:11.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4337 23.63 MB/sec warmup 17 sec latency 1276.913 ms 2024-03-14T21:52:12.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4337 22.32 MB/sec warmup 18 sec latency 2277.004 ms 2024-03-14T21:52:13.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4337 21.14 MB/sec warmup 19 sec latency 3277.072 ms 2024-03-14T21:52:14.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4337 20.09 MB/sec warmup 20 sec latency 4277.181 ms 2024-03-14T21:52:15.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4400 19.22 MB/sec warmup 21 sec latency 5073.787 ms 2024-03-14T21:52:16.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 4843 19.40 MB/sec warmup 22 sec latency 369.325 ms 2024-03-14T21:52:17.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 5510 19.71 MB/sec warmup 23 sec latency 128.474 ms 2024-03-14T21:52:18.285 INFO:tasks.workunit.client.0.smithi163.stdout: 10 6089 20.62 MB/sec warmup 24 sec latency 123.667 ms 2024-03-14T21:52:19.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 6676 21.78 MB/sec warmup 25 sec latency 84.959 ms 2024-03-14T21:52:20.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7011 21.75 MB/sec warmup 26 sec latency 105.805 ms 2024-03-14T21:52:21.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7011 20.94 MB/sec warmup 27 sec latency 1062.009 ms 2024-03-14T21:52:22.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7011 20.19 MB/sec warmup 28 sec latency 2062.098 ms 2024-03-14T21:52:23.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7011 19.50 MB/sec warmup 29 sec latency 3062.180 ms 2024-03-14T21:52:24.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7011 18.85 MB/sec warmup 30 sec latency 4062.259 ms 2024-03-14T21:52:25.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7014 18.24 MB/sec warmup 31 sec latency 5041.908 ms 2024-03-14T21:52:26.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7033 17.68 MB/sec warmup 32 sec latency 852.579 ms 2024-03-14T21:52:27.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7038 17.14 MB/sec warmup 33 sec latency 1103.889 ms 2024-03-14T21:52:28.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7044 16.64 MB/sec warmup 34 sec latency 740.991 ms 2024-03-14T21:52:29.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7057 16.19 MB/sec warmup 35 sec latency 750.479 ms 2024-03-14T21:52:30.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7062 15.74 MB/sec warmup 36 sec latency 785.084 ms 2024-03-14T21:52:31.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7070 15.33 MB/sec warmup 37 sec latency 1114.048 ms 2024-03-14T21:52:32.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7079 14.93 MB/sec warmup 38 sec latency 922.294 ms 2024-03-14T21:52:33.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7081 14.54 MB/sec warmup 39 sec latency 1922.338 ms 2024-03-14T21:52:34.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7081 14.18 MB/sec warmup 40 sec latency 2922.382 ms 2024-03-14T21:52:35.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7081 13.83 MB/sec warmup 41 sec latency 3922.426 ms 2024-03-14T21:52:36.286 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7081 13.50 MB/sec warmup 42 sec latency 4922.470 ms 2024-03-14T21:52:37.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7085 13.19 MB/sec warmup 43 sec latency 5901.477 ms 2024-03-14T21:52:38.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7088 12.89 MB/sec warmup 44 sec latency 1480.283 ms 2024-03-14T21:52:39.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7092 12.61 MB/sec warmup 45 sec latency 1112.146 ms 2024-03-14T21:52:39.715 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:52:39.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:52:39.720 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:52:40.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7100 12.33 MB/sec warmup 46 sec latency 748.578 ms 2024-03-14T21:52:41.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7105 12.07 MB/sec warmup 47 sec latency 740.848 ms 2024-03-14T21:52:42.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7116 11.82 MB/sec warmup 48 sec latency 1104.698 ms 2024-03-14T21:52:43.287 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7118 11.58 MB/sec warmup 49 sec latency 1193.767 ms 2024-03-14T21:52:44.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7118 11.35 MB/sec warmup 50 sec latency 2197.431 ms 2024-03-14T21:52:45.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7118 11.13 MB/sec warmup 51 sec latency 3197.471 ms 2024-03-14T21:52:46.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7118 10.91 MB/sec warmup 52 sec latency 4197.537 ms 2024-03-14T21:52:47.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7118 10.71 MB/sec warmup 53 sec latency 5197.629 ms 2024-03-14T21:52:48.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7122 10.51 MB/sec warmup 54 sec latency 5875.554 ms 2024-03-14T21:52:49.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7123 10.32 MB/sec warmup 55 sec latency 1678.427 ms 2024-03-14T21:52:50.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7128 10.14 MB/sec warmup 56 sec latency 2307.524 ms 2024-03-14T21:52:51.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7134 9.96 MB/sec warmup 57 sec latency 2539.138 ms 2024-03-14T21:52:52.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7139 9.79 MB/sec warmup 58 sec latency 1093.726 ms 2024-03-14T21:52:53.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7147 9.63 MB/sec warmup 59 sec latency 1456.906 ms 2024-03-14T21:52:54.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7150 9.47 MB/sec warmup 60 sec latency 1449.593 ms 2024-03-14T21:52:55.291 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7150 9.31 MB/sec warmup 61 sec latency 1849.588 ms 2024-03-14T21:52:56.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7150 9.16 MB/sec warmup 62 sec latency 2849.619 ms 2024-03-14T21:52:57.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7150 9.02 MB/sec warmup 63 sec latency 3849.648 ms 2024-03-14T21:52:58.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7150 8.88 MB/sec warmup 64 sec latency 4849.704 ms 2024-03-14T21:52:59.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7156 8.74 MB/sec warmup 65 sec latency 5819.557 ms 2024-03-14T21:53:00.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7159 8.61 MB/sec warmup 66 sec latency 1092.709 ms 2024-03-14T21:53:01.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7163 8.48 MB/sec warmup 67 sec latency 1096.113 ms 2024-03-14T21:53:02.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7170 8.36 MB/sec warmup 68 sec latency 1453.265 ms 2024-03-14T21:53:03.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7177 8.25 MB/sec warmup 69 sec latency 1458.787 ms 2024-03-14T21:53:04.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7180 8.13 MB/sec warmup 70 sec latency 1093.038 ms 2024-03-14T21:53:05.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7181 8.02 MB/sec warmup 71 sec latency 1658.933 ms 2024-03-14T21:53:06.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7181 7.91 MB/sec warmup 72 sec latency 2658.974 ms 2024-03-14T21:53:07.292 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7181 7.80 MB/sec warmup 73 sec latency 3659.056 ms 2024-03-14T21:53:08.294 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7181 7.69 MB/sec warmup 74 sec latency 4661.103 ms 2024-03-14T21:53:09.294 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7181 7.59 MB/sec warmup 75 sec latency 5661.164 ms 2024-03-14T21:53:09.818 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:53:09.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:53:09.823 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:53:10.294 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7185 7.49 MB/sec warmup 76 sec latency 6551.434 ms 2024-03-14T21:53:11.294 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7190 7.40 MB/sec warmup 77 sec latency 1098.743 ms 2024-03-14T21:53:12.294 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7191 7.30 MB/sec warmup 78 sec latency 1456.883 ms 2024-03-14T21:53:13.294 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7198 7.21 MB/sec warmup 79 sec latency 1454.844 ms 2024-03-14T21:53:14.294 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7206 7.12 MB/sec warmup 80 sec latency 1090.419 ms 2024-03-14T21:53:15.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7214 7.04 MB/sec warmup 81 sec latency 1089.092 ms 2024-03-14T21:53:16.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7215 6.95 MB/sec warmup 82 sec latency 1737.728 ms 2024-03-14T21:53:17.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7218 6.87 MB/sec warmup 83 sec latency 2737.763 ms 2024-03-14T21:53:18.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7218 6.79 MB/sec warmup 84 sec latency 3737.792 ms 2024-03-14T21:53:19.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7218 6.71 MB/sec warmup 85 sec latency 4737.820 ms 2024-03-14T21:53:20.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7218 6.63 MB/sec warmup 86 sec latency 5737.846 ms 2024-03-14T21:53:21.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7220 6.55 MB/sec warmup 87 sec latency 5836.777 ms 2024-03-14T21:53:22.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7229 6.49 MB/sec warmup 88 sec latency 4369.436 ms 2024-03-14T21:53:23.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7240 6.44 MB/sec warmup 89 sec latency 1093.407 ms 2024-03-14T21:53:24.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7244 6.37 MB/sec warmup 90 sec latency 1356.419 ms 2024-03-14T21:53:25.295 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7247 6.30 MB/sec warmup 91 sec latency 1453.358 ms 2024-03-14T21:53:26.301 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7252 6.23 MB/sec warmup 92 sec latency 1822.571 ms 2024-03-14T21:53:27.301 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7259 6.17 MB/sec warmup 93 sec latency 1094.230 ms 2024-03-14T21:53:28.304 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7260 6.10 MB/sec warmup 94 sec latency 2089.854 ms 2024-03-14T21:53:29.304 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7260 6.04 MB/sec warmup 95 sec latency 3089.887 ms 2024-03-14T21:53:30.307 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7260 5.98 MB/sec warmup 96 sec latency 4092.830 ms 2024-03-14T21:53:31.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7260 5.91 MB/sec warmup 97 sec latency 5100.053 ms 2024-03-14T21:53:32.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7263 5.85 MB/sec warmup 98 sec latency 5831.628 ms 2024-03-14T21:53:33.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7268 5.79 MB/sec warmup 99 sec latency 5469.631 ms 2024-03-14T21:53:34.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7277 5.74 MB/sec warmup 100 sec latency 1456.966 ms 2024-03-14T21:53:35.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7283 5.68 MB/sec warmup 101 sec latency 1086.522 ms 2024-03-14T21:53:36.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7288 5.63 MB/sec warmup 102 sec latency 1093.021 ms 2024-03-14T21:53:37.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7292 5.58 MB/sec warmup 103 sec latency 1455.290 ms 2024-03-14T21:53:38.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7292 5.52 MB/sec warmup 104 sec latency 1898.109 ms 2024-03-14T21:53:39.315 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7292 5.47 MB/sec warmup 105 sec latency 2898.144 ms 2024-03-14T21:53:39.958 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:53:39.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:53:39.963 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:53:40.316 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7292 5.42 MB/sec warmup 106 sec latency 3898.227 ms 2024-03-14T21:53:41.316 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7292 5.37 MB/sec warmup 107 sec latency 4898.294 ms 2024-03-14T21:53:42.323 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7293 5.32 MB/sec warmup 108 sec latency 5822.555 ms 2024-03-14T21:53:43.324 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7301 5.27 MB/sec warmup 109 sec latency 5822.500 ms 2024-03-14T21:53:44.324 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7320 5.25 MB/sec warmup 110 sec latency 1100.339 ms 2024-03-14T21:53:45.324 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7327 5.20 MB/sec warmup 111 sec latency 1097.681 ms 2024-03-14T21:53:46.324 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7332 5.16 MB/sec warmup 112 sec latency 1525.039 ms 2024-03-14T21:53:47.324 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7334 5.11 MB/sec warmup 113 sec latency 2190.669 ms 2024-03-14T21:53:48.325 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7342 5.07 MB/sec warmup 114 sec latency 1822.567 ms 2024-03-14T21:53:49.325 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7344 5.02 MB/sec warmup 115 sec latency 1972.173 ms 2024-03-14T21:53:50.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7344 4.98 MB/sec warmup 116 sec latency 2972.210 ms 2024-03-14T21:53:51.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7344 4.94 MB/sec warmup 117 sec latency 3972.257 ms 2024-03-14T21:53:52.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7344 4.90 MB/sec warmup 118 sec latency 4972.309 ms 2024-03-14T21:53:53.327 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7344 4.85 MB/sec warmup 119 sec latency 5972.360 ms 2024-03-14T21:53:55.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7357 0.02 MB/sec execute 1 sec latency 6201.637 ms 2024-03-14T21:53:56.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7362 0.02 MB/sec execute 2 sec latency 1453.739 ms 2024-03-14T21:53:57.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7369 0.03 MB/sec execute 3 sec latency 1458.914 ms 2024-03-14T21:53:58.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7376 0.03 MB/sec execute 4 sec latency 1455.035 ms 2024-03-14T21:53:59.326 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7380 0.03 MB/sec execute 5 sec latency 1399.116 ms 2024-03-14T21:54:00.328 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7389 0.05 MB/sec execute 6 sec latency 1455.822 ms 2024-03-14T21:54:01.328 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7401 0.19 MB/sec execute 7 sec latency 1214.209 ms 2024-03-14T21:54:02.328 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7404 0.17 MB/sec execute 8 sec latency 2181.566 ms 2024-03-14T21:54:03.331 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7409 0.16 MB/sec execute 9 sec latency 1819.266 ms 2024-03-14T21:54:04.331 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7413 0.14 MB/sec execute 10 sec latency 1817.311 ms 2024-03-14T21:54:05.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7419 0.13 MB/sec execute 11 sec latency 1827.106 ms 2024-03-14T21:54:06.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7424 0.12 MB/sec execute 12 sec latency 1821.431 ms 2024-03-14T21:54:07.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7433 0.13 MB/sec execute 13 sec latency 1096.004 ms 2024-03-14T21:54:08.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7441 0.12 MB/sec execute 14 sec latency 1461.872 ms 2024-03-14T21:54:09.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7447 0.12 MB/sec execute 15 sec latency 1095.130 ms 2024-03-14T21:54:10.046 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:54:10.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:54:10.050 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:54:10.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7458 0.12 MB/sec execute 16 sec latency 1097.174 ms 2024-03-14T21:54:11.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7466 0.11 MB/sec execute 17 sec latency 1453.420 ms 2024-03-14T21:54:12.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7475 0.11 MB/sec execute 18 sec latency 924.068 ms 2024-03-14T21:54:13.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7481 0.11 MB/sec execute 19 sec latency 1090.605 ms 2024-03-14T21:54:14.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7493 0.10 MB/sec execute 20 sec latency 1817.508 ms 2024-03-14T21:54:15.332 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7497 0.10 MB/sec execute 21 sec latency 1087.965 ms 2024-03-14T21:54:16.333 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7504 0.10 MB/sec execute 22 sec latency 1089.995 ms 2024-03-14T21:54:17.333 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7517 0.10 MB/sec execute 23 sec latency 1094.919 ms 2024-03-14T21:54:18.333 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7534 0.22 MB/sec execute 24 sec latency 1813.218 ms 2024-03-14T21:54:19.334 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7536 0.21 MB/sec execute 25 sec latency 1091.023 ms 2024-03-14T21:54:20.334 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7560 0.21 MB/sec execute 26 sec latency 1089.731 ms 2024-03-14T21:54:21.334 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7566 0.21 MB/sec execute 27 sec latency 1096.573 ms 2024-03-14T21:54:22.334 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7578 0.20 MB/sec execute 28 sec latency 1097.778 ms 2024-03-14T21:54:23.335 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7587 0.20 MB/sec execute 29 sec latency 1458.461 ms 2024-03-14T21:54:24.335 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7610 0.29 MB/sec execute 30 sec latency 913.783 ms 2024-03-14T21:54:25.335 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7626 0.33 MB/sec execute 31 sec latency 1825.858 ms 2024-03-14T21:54:26.335 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7633 0.32 MB/sec execute 32 sec latency 1092.310 ms 2024-03-14T21:54:27.335 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7643 0.31 MB/sec execute 33 sec latency 1454.288 ms 2024-03-14T21:54:28.338 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7650 0.30 MB/sec execute 34 sec latency 1638.016 ms 2024-03-14T21:54:29.336 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7661 0.30 MB/sec execute 35 sec latency 1819.202 ms 2024-03-14T21:54:30.336 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7670 0.29 MB/sec execute 36 sec latency 1090.397 ms 2024-03-14T21:54:31.336 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7677 0.29 MB/sec execute 37 sec latency 1727.397 ms 2024-03-14T21:54:32.342 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7691 0.28 MB/sec execute 38 sec latency 2185.375 ms 2024-03-14T21:54:33.342 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7700 0.28 MB/sec execute 39 sec latency 1458.309 ms 2024-03-14T21:54:34.342 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7704 0.27 MB/sec execute 40 sec latency 1816.227 ms 2024-03-14T21:54:35.342 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7712 0.27 MB/sec execute 41 sec latency 2816.282 ms 2024-03-14T21:54:36.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7723 0.27 MB/sec execute 42 sec latency 3816.319 ms 2024-03-14T21:54:37.345 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7739 0.27 MB/sec execute 43 sec latency 4001.229 ms 2024-03-14T21:54:38.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7742 0.26 MB/sec execute 44 sec latency 1457.495 ms 2024-03-14T21:54:39.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7750 0.26 MB/sec execute 45 sec latency 2192.000 ms 2024-03-14T21:54:40.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7757 0.26 MB/sec execute 46 sec latency 1094.337 ms 2024-03-14T21:54:40.348 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:54:40.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:54:40.352 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:54:41.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7769 0.26 MB/sec execute 47 sec latency 1093.789 ms 2024-03-14T21:54:42.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7776 0.25 MB/sec execute 48 sec latency 1076.377 ms 2024-03-14T21:54:43.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7783 0.25 MB/sec execute 49 sec latency 1097.563 ms 2024-03-14T21:54:44.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7804 0.25 MB/sec execute 50 sec latency 1615.667 ms 2024-03-14T21:54:45.343 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7809 0.24 MB/sec execute 51 sec latency 2615.702 ms 2024-03-14T21:54:46.347 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7814 0.25 MB/sec execute 52 sec latency 3615.737 ms 2024-03-14T21:54:47.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7827 0.25 MB/sec execute 53 sec latency 4615.778 ms 2024-03-14T21:54:48.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7839 0.27 MB/sec execute 54 sec latency 4722.268 ms 2024-03-14T21:54:49.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7851 0.27 MB/sec execute 55 sec latency 1888.648 ms 2024-03-14T21:54:50.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7857 0.26 MB/sec execute 56 sec latency 2173.348 ms 2024-03-14T21:54:51.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7867 0.26 MB/sec execute 57 sec latency 1450.691 ms 2024-03-14T21:54:52.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7887 0.26 MB/sec execute 58 sec latency 1093.896 ms 2024-03-14T21:54:53.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7896 0.26 MB/sec execute 59 sec latency 1458.966 ms 2024-03-14T21:54:54.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7896 0.26 MB/sec execute 60 sec latency 2164.726 ms 2024-03-14T21:54:55.344 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7896 0.25 MB/sec execute 61 sec latency 3164.756 ms 2024-03-14T21:54:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7896 0.25 MB/sec execute 62 sec latency 4164.787 ms 2024-03-14T21:54:57.348 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7896 0.24 MB/sec execute 63 sec latency 5168.498 ms 2024-03-14T21:54:58.348 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7900 0.24 MB/sec execute 64 sec latency 5812.054 ms 2024-03-14T21:54:59.349 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7903 0.24 MB/sec execute 65 sec latency 6539.937 ms 2024-03-14T21:55:00.349 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7932 0.24 MB/sec execute 66 sec latency 1460.041 ms 2024-03-14T21:55:01.349 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7961 0.26 MB/sec execute 67 sec latency 1095.218 ms 2024-03-14T21:55:02.349 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7969 0.26 MB/sec execute 68 sec latency 1075.645 ms 2024-03-14T21:55:03.349 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7984 0.26 MB/sec execute 69 sec latency 1092.804 ms 2024-03-14T21:55:04.349 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7985 0.25 MB/sec execute 70 sec latency 1976.944 ms 2024-03-14T21:55:05.349 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7986 0.25 MB/sec execute 71 sec latency 2976.980 ms 2024-03-14T21:55:06.356 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7986 0.25 MB/sec execute 72 sec latency 3983.421 ms 2024-03-14T21:55:07.358 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7986 0.24 MB/sec execute 73 sec latency 4985.662 ms 2024-03-14T21:55:08.358 INFO:tasks.workunit.client.0.smithi163.stdout: 10 7996 0.24 MB/sec execute 74 sec latency 5813.668 ms 2024-03-14T21:55:09.358 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8013 0.25 MB/sec execute 75 sec latency 6188.344 ms 2024-03-14T21:55:10.358 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8033 0.30 MB/sec execute 76 sec latency 1438.430 ms 2024-03-14T21:55:10.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:55:10.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:55:10.449 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:55:11.358 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8049 0.30 MB/sec execute 77 sec latency 1458.812 ms 2024-03-14T21:55:12.358 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8059 0.30 MB/sec execute 78 sec latency 1090.738 ms 2024-03-14T21:55:13.371 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8066 0.30 MB/sec execute 79 sec latency 1089.875 ms 2024-03-14T21:55:14.371 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8074 0.30 MB/sec execute 80 sec latency 1091.729 ms 2024-03-14T21:55:15.372 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8084 0.30 MB/sec execute 81 sec latency 1354.933 ms 2024-03-14T21:55:16.372 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8084 0.29 MB/sec execute 82 sec latency 2354.964 ms 2024-03-14T21:55:17.372 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8084 0.29 MB/sec execute 83 sec latency 3355.001 ms 2024-03-14T21:55:18.376 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8084 0.29 MB/sec execute 84 sec latency 4359.489 ms 2024-03-14T21:55:19.377 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8084 0.28 MB/sec execute 85 sec latency 5092.572 ms 2024-03-14T21:55:20.377 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8105 0.28 MB/sec execute 86 sec latency 5820.918 ms 2024-03-14T21:55:21.377 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8117 0.28 MB/sec execute 87 sec latency 5453.668 ms 2024-03-14T21:55:22.377 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8125 0.29 MB/sec execute 88 sec latency 1084.091 ms 2024-03-14T21:55:23.380 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8125 0.28 MB/sec execute 89 sec latency 2087.116 ms 2024-03-14T21:55:24.380 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8132 0.28 MB/sec execute 90 sec latency 2547.950 ms 2024-03-14T21:55:25.381 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8137 0.28 MB/sec execute 91 sec latency 1539.765 ms 2024-03-14T21:55:26.381 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8141 0.28 MB/sec execute 92 sec latency 1819.776 ms 2024-03-14T21:55:27.381 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8149 0.30 MB/sec execute 93 sec latency 2084.769 ms 2024-03-14T21:55:28.381 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8168 0.33 MB/sec execute 94 sec latency 3084.809 ms 2024-03-14T21:55:29.381 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8176 0.32 MB/sec execute 95 sec latency 4084.852 ms 2024-03-14T21:55:30.381 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8184 0.32 MB/sec execute 96 sec latency 5084.890 ms 2024-03-14T21:55:31.382 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8190 0.32 MB/sec execute 97 sec latency 5090.954 ms 2024-03-14T21:55:32.382 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8198 0.32 MB/sec execute 98 sec latency 2358.063 ms 2024-03-14T21:55:33.382 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8218 0.32 MB/sec execute 99 sec latency 2555.122 ms 2024-03-14T21:55:34.382 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8225 0.32 MB/sec execute 100 sec latency 1070.650 ms 2024-03-14T21:55:35.382 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8244 0.35 MB/sec execute 101 sec latency 1455.650 ms 2024-03-14T21:55:36.389 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8249 0.34 MB/sec execute 102 sec latency 1824.622 ms 2024-03-14T21:55:37.389 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8255 0.34 MB/sec execute 103 sec latency 1818.895 ms 2024-03-14T21:55:38.389 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8255 0.34 MB/sec execute 104 sec latency 1786.532 ms 2024-03-14T21:55:39.390 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8255 0.34 MB/sec execute 105 sec latency 2787.641 ms 2024-03-14T21:55:40.390 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8255 0.33 MB/sec execute 106 sec latency 3787.685 ms 2024-03-14T21:55:40.776 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:55:40.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:55:40.783 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:55:41.390 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8264 0.33 MB/sec execute 107 sec latency 4757.676 ms 2024-03-14T21:55:42.390 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8276 0.34 MB/sec execute 108 sec latency 1028.423 ms 2024-03-14T21:55:43.391 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8308 0.36 MB/sec execute 109 sec latency 1128.042 ms 2024-03-14T21:55:44.391 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8329 0.36 MB/sec execute 110 sec latency 754.565 ms 2024-03-14T21:55:45.391 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8334 0.36 MB/sec execute 111 sec latency 1043.325 ms 2024-03-14T21:55:46.396 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8348 0.36 MB/sec execute 112 sec latency 1501.535 ms 2024-03-14T21:55:47.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8348 0.35 MB/sec execute 113 sec latency 1945.581 ms 2024-03-14T21:55:48.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8348 0.35 MB/sec execute 114 sec latency 2945.647 ms 2024-03-14T21:55:49.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8348 0.35 MB/sec execute 115 sec latency 3945.708 ms 2024-03-14T21:55:50.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8348 0.34 MB/sec execute 116 sec latency 4945.775 ms 2024-03-14T21:55:51.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8348 0.34 MB/sec execute 117 sec latency 5945.868 ms 2024-03-14T21:55:52.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8355 0.34 MB/sec execute 118 sec latency 6739.618 ms 2024-03-14T21:55:53.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8371 0.34 MB/sec execute 119 sec latency 7097.097 ms 2024-03-14T21:55:54.414 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8376 0.34 MB/sec execute 120 sec latency 1448.543 ms 2024-03-14T21:55:55.415 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8385 0.34 MB/sec execute 121 sec latency 2448.605 ms 2024-03-14T21:55:56.415 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8395 0.34 MB/sec execute 122 sec latency 2615.321 ms 2024-03-14T21:55:57.415 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8403 0.34 MB/sec execute 123 sec latency 1832.780 ms 2024-03-14T21:55:58.418 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8405 0.34 MB/sec execute 124 sec latency 2832.825 ms 2024-03-14T21:55:59.420 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8411 0.34 MB/sec execute 125 sec latency 3838.141 ms 2024-03-14T21:56:00.420 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8411 0.33 MB/sec execute 126 sec latency 4838.207 ms 2024-03-14T21:56:01.420 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8411 0.33 MB/sec execute 127 sec latency 5838.265 ms 2024-03-14T21:56:02.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8412 0.33 MB/sec execute 128 sec latency 6589.773 ms 2024-03-14T21:56:03.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8416 0.33 MB/sec execute 129 sec latency 5819.848 ms 2024-03-14T21:56:04.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8417 0.33 MB/sec execute 130 sec latency 5508.881 ms 2024-03-14T21:56:05.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8418 0.33 MB/sec execute 131 sec latency 6508.933 ms 2024-03-14T21:56:06.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8423 0.32 MB/sec execute 132 sec latency 7508.982 ms 2024-03-14T21:56:07.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8427 0.32 MB/sec execute 133 sec latency 8509.013 ms 2024-03-14T21:56:08.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8430 0.32 MB/sec execute 134 sec latency 9509.040 ms 2024-03-14T21:56:09.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8430 0.32 MB/sec execute 135 sec latency 10509.070 ms 2024-03-14T21:56:10.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8430 0.32 MB/sec execute 136 sec latency 11509.098 ms 2024-03-14T21:56:10.881 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:56:10.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:56:10.887 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:56:11.421 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8430 0.31 MB/sec execute 137 sec latency 12509.132 ms 2024-03-14T21:56:12.429 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8430 0.31 MB/sec execute 138 sec latency 13516.457 ms 2024-03-14T21:56:13.429 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8431 0.31 MB/sec execute 139 sec latency 14516.491 ms 2024-03-14T21:56:14.429 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8436 0.31 MB/sec execute 140 sec latency 15516.552 ms 2024-03-14T21:56:15.432 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8436 0.31 MB/sec execute 141 sec latency 16519.973 ms 2024-03-14T21:56:16.432 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8445 0.32 MB/sec execute 142 sec latency 17520.011 ms 2024-03-14T21:56:17.432 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8446 0.32 MB/sec execute 143 sec latency 18520.046 ms 2024-03-14T21:56:18.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8447 0.32 MB/sec execute 144 sec latency 19223.797 ms 2024-03-14T21:56:19.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8460 0.34 MB/sec execute 145 sec latency 2901.832 ms 2024-03-14T21:56:20.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8464 0.33 MB/sec execute 146 sec latency 2175.582 ms 2024-03-14T21:56:21.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8470 0.33 MB/sec execute 147 sec latency 1845.769 ms 2024-03-14T21:56:22.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8470 0.33 MB/sec execute 148 sec latency 2845.830 ms 2024-03-14T21:56:23.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8470 0.33 MB/sec execute 149 sec latency 3845.888 ms 2024-03-14T21:56:24.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8476 0.33 MB/sec execute 150 sec latency 4717.059 ms 2024-03-14T21:56:25.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8481 0.32 MB/sec execute 151 sec latency 5121.068 ms 2024-03-14T21:56:26.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8482 0.32 MB/sec execute 152 sec latency 5445.200 ms 2024-03-14T21:56:27.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8496 0.33 MB/sec execute 153 sec latency 1814.916 ms 2024-03-14T21:56:28.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8504 0.34 MB/sec execute 154 sec latency 2675.996 ms 2024-03-14T21:56:29.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8508 0.34 MB/sec execute 155 sec latency 3676.033 ms 2024-03-14T21:56:30.433 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8508 0.33 MB/sec execute 156 sec latency 4676.086 ms 2024-03-14T21:56:31.434 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8531 0.34 MB/sec execute 157 sec latency 4714.901 ms 2024-03-14T21:56:32.434 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8535 0.34 MB/sec execute 158 sec latency 1960.874 ms 2024-03-14T21:56:33.434 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8547 0.35 MB/sec execute 159 sec latency 2196.650 ms 2024-03-14T21:56:34.437 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8548 0.35 MB/sec execute 160 sec latency 1769.025 ms 2024-03-14T21:56:35.437 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8577 0.36 MB/sec execute 161 sec latency 2767.651 ms 2024-03-14T21:56:36.437 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8592 0.36 MB/sec execute 162 sec latency 3266.977 ms 2024-03-14T21:56:37.437 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8597 0.37 MB/sec execute 163 sec latency 1861.046 ms 2024-03-14T21:56:38.437 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8605 0.38 MB/sec execute 164 sec latency 2534.879 ms 2024-03-14T21:56:39.437 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8617 0.38 MB/sec execute 165 sec latency 3263.285 ms 2024-03-14T21:56:40.442 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8622 0.37 MB/sec execute 166 sec latency 1454.632 ms 2024-03-14T21:56:40.980 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:56:40.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:56:40.985 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:56:41.442 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8622 0.37 MB/sec execute 167 sec latency 2242.503 ms 2024-03-14T21:56:42.442 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8622 0.37 MB/sec execute 168 sec latency 3242.595 ms 2024-03-14T21:56:43.442 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8622 0.37 MB/sec execute 169 sec latency 4242.713 ms 2024-03-14T21:56:44.443 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8622 0.37 MB/sec execute 170 sec latency 5242.791 ms 2024-03-14T21:56:45.443 INFO:tasks.workunit.client.0.smithi163.stdout: 10 8824 0.41 MB/sec execute 171 sec latency 5728.493 ms 2024-03-14T21:56:46.443 INFO:tasks.workunit.client.0.smithi163.stdout: 10 9386 0.55 MB/sec execute 172 sec latency 83.664 ms 2024-03-14T21:56:47.443 INFO:tasks.workunit.client.0.smithi163.stdout: 10 9619 0.64 MB/sec execute 173 sec latency 405.744 ms 2024-03-14T21:56:48.443 INFO:tasks.workunit.client.0.smithi163.stdout: 10 9983 0.79 MB/sec execute 174 sec latency 164.194 ms 2024-03-14T21:56:49.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10394 0.95 MB/sec execute 175 sec latency 92.904 ms 2024-03-14T21:56:50.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10515 0.97 MB/sec execute 176 sec latency 351.684 ms 2024-03-14T21:56:51.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10515 0.97 MB/sec execute 177 sec latency 1351.787 ms 2024-03-14T21:56:52.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10515 0.96 MB/sec execute 178 sec latency 2351.861 ms 2024-03-14T21:56:53.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10515 0.96 MB/sec execute 179 sec latency 3351.960 ms 2024-03-14T21:56:54.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10515 0.95 MB/sec execute 180 sec latency 4352.070 ms 2024-03-14T21:56:55.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10625 0.97 MB/sec execute 181 sec latency 5009.847 ms 2024-03-14T21:56:56.444 INFO:tasks.workunit.client.0.smithi163.stdout: 10 10948 1.02 MB/sec execute 182 sec latency 98.847 ms 2024-03-14T21:56:57.445 INFO:tasks.workunit.client.0.smithi163.stdout: 10 11264 1.08 MB/sec execute 183 sec latency 148.067 ms 2024-03-14T21:56:58.445 INFO:tasks.workunit.client.0.smithi163.stdout: 10 11579 1.16 MB/sec execute 184 sec latency 161.279 ms 2024-03-14T21:56:59.445 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12069 1.32 MB/sec execute 185 sec latency 131.158 ms 2024-03-14T21:57:00.445 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12630 1.45 MB/sec execute 186 sec latency 98.885 ms 2024-03-14T21:57:01.445 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12822 1.48 MB/sec execute 187 sec latency 619.634 ms 2024-03-14T21:57:02.445 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12822 1.47 MB/sec execute 188 sec latency 1619.747 ms 2024-03-14T21:57:03.445 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12822 1.47 MB/sec execute 189 sec latency 2619.851 ms 2024-03-14T21:57:04.448 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12822 1.46 MB/sec execute 190 sec latency 3622.194 ms 2024-03-14T21:57:05.448 INFO:tasks.workunit.client.0.smithi163.stdout: 10 12822 1.45 MB/sec execute 191 sec latency 4622.300 ms 2024-03-14T21:57:06.448 INFO:tasks.workunit.client.0.smithi163.stdout: 10 13034 1.51 MB/sec execute 192 sec latency 5031.486 ms 2024-03-14T21:57:07.448 INFO:tasks.workunit.client.0.smithi163.stdout: 10 13605 1.72 MB/sec execute 193 sec latency 105.524 ms 2024-03-14T21:57:08.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14017 1.83 MB/sec execute 194 sec latency 88.200 ms 2024-03-14T21:57:09.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14361 1.89 MB/sec execute 195 sec latency 107.868 ms 2024-03-14T21:57:10.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14720 1.97 MB/sec execute 196 sec latency 165.870 ms 2024-03-14T21:57:11.412 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:57:11.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:57:11.417 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:57:11.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14859 1.99 MB/sec execute 197 sec latency 406.855 ms 2024-03-14T21:57:12.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14859 1.98 MB/sec execute 198 sec latency 1406.939 ms 2024-03-14T21:57:13.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14859 1.97 MB/sec execute 199 sec latency 2407.033 ms 2024-03-14T21:57:14.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14859 1.96 MB/sec execute 200 sec latency 3407.135 ms 2024-03-14T21:57:15.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14859 1.95 MB/sec execute 201 sec latency 4407.214 ms 2024-03-14T21:57:16.449 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14892 1.94 MB/sec execute 202 sec latency 5044.509 ms 2024-03-14T21:57:17.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14900 1.94 MB/sec execute 203 sec latency 1182.405 ms 2024-03-14T21:57:18.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14902 1.93 MB/sec execute 204 sec latency 1147.194 ms 2024-03-14T21:57:19.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14904 1.92 MB/sec execute 205 sec latency 2147.233 ms 2024-03-14T21:57:20.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14908 1.91 MB/sec execute 206 sec latency 3136.886 ms 2024-03-14T21:57:21.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14922 1.90 MB/sec execute 207 sec latency 3987.135 ms 2024-03-14T21:57:22.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14932 1.90 MB/sec execute 208 sec latency 1088.903 ms 2024-03-14T21:57:23.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14933 1.89 MB/sec execute 209 sec latency 1786.113 ms 2024-03-14T21:57:24.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14933 1.88 MB/sec execute 210 sec latency 2786.147 ms 2024-03-14T21:57:25.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14934 1.87 MB/sec execute 211 sec latency 3786.177 ms 2024-03-14T21:57:26.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14934 1.86 MB/sec execute 212 sec latency 4786.207 ms 2024-03-14T21:57:27.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14935 1.85 MB/sec execute 213 sec latency 5786.238 ms 2024-03-14T21:57:28.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14937 1.84 MB/sec execute 214 sec latency 6176.698 ms 2024-03-14T21:57:29.450 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14946 1.84 MB/sec execute 215 sec latency 7059.798 ms 2024-03-14T21:57:30.451 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14957 1.83 MB/sec execute 216 sec latency 8059.839 ms 2024-03-14T21:57:31.452 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14962 1.82 MB/sec execute 217 sec latency 8727.468 ms 2024-03-14T21:57:32.453 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14970 1.82 MB/sec execute 218 sec latency 1452.699 ms 2024-03-14T21:57:33.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14983 1.81 MB/sec execute 219 sec latency 2176.249 ms 2024-03-14T21:57:34.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14990 1.81 MB/sec execute 220 sec latency 2174.123 ms 2024-03-14T21:57:35.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14993 1.80 MB/sec execute 221 sec latency 1440.284 ms 2024-03-14T21:57:36.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 14999 1.79 MB/sec execute 222 sec latency 2440.316 ms 2024-03-14T21:57:37.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15003 1.78 MB/sec execute 223 sec latency 3440.345 ms 2024-03-14T21:57:38.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15004 1.77 MB/sec execute 224 sec latency 4440.373 ms 2024-03-14T21:57:39.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15015 1.77 MB/sec execute 225 sec latency 4718.322 ms 2024-03-14T21:57:40.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15029 1.76 MB/sec execute 226 sec latency 1462.970 ms 2024-03-14T21:57:41.459 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15036 1.75 MB/sec execute 227 sec latency 1826.218 ms 2024-03-14T21:57:42.481 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15037 1.74 MB/sec execute 228 sec latency 1546.966 ms 2024-03-14T21:57:43.481 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15037 1.74 MB/sec execute 229 sec latency 2547.105 ms 2024-03-14T21:57:44.482 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15046 1.73 MB/sec execute 230 sec latency 2935.068 ms 2024-03-14T21:57:45.482 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15055 1.72 MB/sec execute 231 sec latency 1231.117 ms 2024-03-14T21:57:46.220 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:57:46.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:57:46.225 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:57:46.482 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15059 1.72 MB/sec execute 232 sec latency 2231.170 ms 2024-03-14T21:57:47.482 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15059 1.71 MB/sec execute 233 sec latency 3231.204 ms 2024-03-14T21:57:48.482 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15059 1.70 MB/sec execute 234 sec latency 4231.237 ms 2024-03-14T21:57:49.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15061 1.70 MB/sec execute 235 sec latency 5085.203 ms 2024-03-14T21:57:50.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15064 1.69 MB/sec execute 236 sec latency 5082.571 ms 2024-03-14T21:57:51.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15067 1.68 MB/sec execute 237 sec latency 5775.970 ms 2024-03-14T21:57:52.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15072 1.68 MB/sec execute 238 sec latency 5799.798 ms 2024-03-14T21:57:53.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15080 1.68 MB/sec execute 239 sec latency 4149.839 ms 2024-03-14T21:57:54.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15082 1.67 MB/sec execute 240 sec latency 5074.642 ms 2024-03-14T21:57:55.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15082 1.67 MB/sec execute 241 sec latency 3250.779 ms 2024-03-14T21:57:56.486 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15082 1.66 MB/sec execute 242 sec latency 4250.810 ms 2024-03-14T21:57:57.493 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15082 1.65 MB/sec execute 243 sec latency 5258.118 ms 2024-03-14T21:57:58.493 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15082 1.65 MB/sec execute 244 sec latency 6258.175 ms 2024-03-14T21:57:59.505 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15083 1.64 MB/sec execute 245 sec latency 7269.622 ms 2024-03-14T21:58:00.505 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15084 1.63 MB/sec execute 246 sec latency 8269.661 ms 2024-03-14T21:58:01.505 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15087 1.63 MB/sec execute 247 sec latency 9072.339 ms 2024-03-14T21:58:02.505 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15098 1.62 MB/sec execute 248 sec latency 9064.684 ms 2024-03-14T21:58:03.505 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15115 1.63 MB/sec execute 249 sec latency 3275.880 ms 2024-03-14T21:58:04.505 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15119 1.62 MB/sec execute 250 sec latency 1745.673 ms 2024-03-14T21:58:05.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15127 1.62 MB/sec execute 251 sec latency 1815.623 ms 2024-03-14T21:58:06.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15127 1.61 MB/sec execute 252 sec latency 2655.518 ms 2024-03-14T21:58:07.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15127 1.60 MB/sec execute 253 sec latency 3655.584 ms 2024-03-14T21:58:08.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15127 1.60 MB/sec execute 254 sec latency 4655.640 ms 2024-03-14T21:58:09.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15127 1.59 MB/sec execute 255 sec latency 5655.674 ms 2024-03-14T21:58:10.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15129 1.58 MB/sec execute 256 sec latency 6217.432 ms 2024-03-14T21:58:11.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15132 1.58 MB/sec execute 257 sec latency 6930.530 ms 2024-03-14T21:58:12.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15151 1.58 MB/sec execute 258 sec latency 7315.997 ms 2024-03-14T21:58:13.506 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15169 1.58 MB/sec execute 259 sec latency 839.864 ms 2024-03-14T21:58:14.507 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15180 1.57 MB/sec execute 260 sec latency 1473.411 ms 2024-03-14T21:58:15.507 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15183 1.57 MB/sec execute 261 sec latency 1468.405 ms 2024-03-14T21:58:16.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:58:16.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:58:16.313 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:58:16.511 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15183 1.56 MB/sec execute 262 sec latency 2370.812 ms 2024-03-14T21:58:17.511 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15183 1.56 MB/sec execute 263 sec latency 3370.870 ms 2024-03-14T21:58:18.511 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15183 1.55 MB/sec execute 264 sec latency 4370.899 ms 2024-03-14T21:58:19.511 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15183 1.54 MB/sec execute 265 sec latency 5370.925 ms 2024-03-14T21:58:20.513 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15183 1.54 MB/sec execute 266 sec latency 6370.952 ms 2024-03-14T21:58:21.511 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15183 1.53 MB/sec execute 267 sec latency 7370.981 ms 2024-03-14T21:58:22.511 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15187 1.53 MB/sec execute 268 sec latency 8371.018 ms 2024-03-14T21:58:23.511 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15188 1.52 MB/sec execute 269 sec latency 9371.115 ms 2024-03-14T21:58:24.512 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15200 1.52 MB/sec execute 270 sec latency 9807.379 ms 2024-03-14T21:58:25.512 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15202 1.51 MB/sec execute 271 sec latency 1563.293 ms 2024-03-14T21:58:26.512 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15206 1.51 MB/sec execute 272 sec latency 2563.335 ms 2024-03-14T21:58:27.512 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15211 1.51 MB/sec execute 273 sec latency 3272.869 ms 2024-03-14T21:58:28.512 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15218 1.51 MB/sec execute 274 sec latency 1290.258 ms 2024-03-14T21:58:29.526 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15226 1.50 MB/sec execute 275 sec latency 2174.273 ms 2024-03-14T21:58:30.526 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15229 1.50 MB/sec execute 276 sec latency 1492.375 ms 2024-03-14T21:58:31.530 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15231 1.49 MB/sec execute 277 sec latency 2172.261 ms 2024-03-14T21:58:32.534 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15241 1.49 MB/sec execute 278 sec latency 2536.404 ms 2024-03-14T21:58:33.534 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15257 1.49 MB/sec execute 279 sec latency 1818.327 ms 2024-03-14T21:58:34.534 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15259 1.49 MB/sec execute 280 sec latency 1871.446 ms 2024-03-14T21:58:35.534 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15260 1.48 MB/sec execute 281 sec latency 2868.100 ms 2024-03-14T21:58:36.534 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15264 1.48 MB/sec execute 282 sec latency 2895.511 ms 2024-03-14T21:58:37.534 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15265 1.47 MB/sec execute 283 sec latency 1972.310 ms 2024-03-14T21:58:38.546 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15268 1.47 MB/sec execute 284 sec latency 2905.349 ms 2024-03-14T21:58:39.547 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15268 1.46 MB/sec execute 285 sec latency 3257.588 ms 2024-03-14T21:58:40.547 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15275 1.46 MB/sec execute 286 sec latency 3623.384 ms 2024-03-14T21:58:41.547 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15283 1.45 MB/sec execute 287 sec latency 1089.250 ms 2024-03-14T21:58:42.553 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15286 1.45 MB/sec execute 288 sec latency 1913.515 ms 2024-03-14T21:58:43.553 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15287 1.44 MB/sec execute 289 sec latency 2913.550 ms 2024-03-14T21:58:44.555 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15291 1.44 MB/sec execute 290 sec latency 3264.211 ms 2024-03-14T21:58:45.555 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15303 1.43 MB/sec execute 291 sec latency 3263.506 ms 2024-03-14T21:58:46.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:58:46.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:58:46.470 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:58:46.555 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15306 1.43 MB/sec execute 292 sec latency 3014.603 ms 2024-03-14T21:58:47.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15306 1.42 MB/sec execute 293 sec latency 4014.640 ms 2024-03-14T21:58:48.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15307 1.42 MB/sec execute 294 sec latency 5014.677 ms 2024-03-14T21:58:49.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15327 1.42 MB/sec execute 295 sec latency 5096.907 ms 2024-03-14T21:58:50.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15327 1.41 MB/sec execute 296 sec latency 1924.925 ms 2024-03-14T21:58:51.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15327 1.41 MB/sec execute 297 sec latency 2924.955 ms 2024-03-14T21:58:52.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15327 1.40 MB/sec execute 298 sec latency 3924.985 ms 2024-03-14T21:58:53.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15679 1.46 MB/sec execute 299 sec latency 4265.796 ms 2024-03-14T21:58:54.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15979 1.50 MB/sec execute 300 sec latency 554.281 ms 2024-03-14T21:58:55.556 INFO:tasks.workunit.client.0.smithi163.stdout: 10 15987 1.49 MB/sec execute 301 sec latency 823.344 ms 2024-03-14T21:58:56.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16144 1.51 MB/sec execute 302 sec latency 1104.505 ms 2024-03-14T21:58:57.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16476 1.59 MB/sec execute 303 sec latency 300.344 ms 2024-03-14T21:58:58.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16733 1.64 MB/sec execute 304 sec latency 535.047 ms 2024-03-14T21:58:59.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16733 1.64 MB/sec execute 305 sec latency 1535.092 ms 2024-03-14T21:59:00.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16733 1.63 MB/sec execute 306 sec latency 2535.121 ms 2024-03-14T21:59:01.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16733 1.63 MB/sec execute 307 sec latency 3535.148 ms 2024-03-14T21:59:02.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16733 1.62 MB/sec execute 308 sec latency 4535.179 ms 2024-03-14T21:59:03.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16736 1.62 MB/sec execute 309 sec latency 5535.246 ms 2024-03-14T21:59:04.557 INFO:tasks.workunit.client.0.smithi163.stdout: 10 16868 1.64 MB/sec execute 310 sec latency 5849.417 ms 2024-03-14T21:59:05.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 17327 1.73 MB/sec execute 311 sec latency 105.762 ms 2024-03-14T21:59:06.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 17675 1.77 MB/sec execute 312 sec latency 98.728 ms 2024-03-14T21:59:07.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 17987 1.81 MB/sec execute 313 sec latency 166.255 ms 2024-03-14T21:59:08.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18263 1.84 MB/sec execute 314 sec latency 150.373 ms 2024-03-14T21:59:09.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18395 1.86 MB/sec execute 315 sec latency 705.057 ms 2024-03-14T21:59:10.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18395 1.86 MB/sec execute 316 sec latency 1705.177 ms 2024-03-14T21:59:11.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18395 1.85 MB/sec execute 317 sec latency 2705.294 ms 2024-03-14T21:59:12.558 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18395 1.85 MB/sec execute 318 sec latency 3705.370 ms 2024-03-14T21:59:13.559 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18395 1.84 MB/sec execute 319 sec latency 4705.449 ms 2024-03-14T21:59:14.559 INFO:tasks.workunit.client.0.smithi163.stdout: 10 18759 1.90 MB/sec execute 320 sec latency 5002.788 ms 2024-03-14T21:59:15.559 INFO:tasks.workunit.client.0.smithi163.stdout: 10 19316 1.96 MB/sec execute 321 sec latency 98.604 ms 2024-03-14T21:59:16.559 INFO:tasks.workunit.client.0.smithi163.stdout: 10 19721 2.02 MB/sec execute 322 sec latency 109.854 ms 2024-03-14T21:59:16.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:59:16.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:59:16.642 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:59:17.559 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20152 2.12 MB/sec execute 323 sec latency 124.566 ms 2024-03-14T21:59:18.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20624 2.20 MB/sec execute 324 sec latency 112.202 ms 2024-03-14T21:59:19.559 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20824 2.21 MB/sec execute 325 sec latency 479.515 ms 2024-03-14T21:59:20.559 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20824 2.20 MB/sec execute 326 sec latency 1479.606 ms 2024-03-14T21:59:21.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20824 2.19 MB/sec execute 327 sec latency 2479.685 ms 2024-03-14T21:59:22.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20824 2.19 MB/sec execute 328 sec latency 3479.780 ms 2024-03-14T21:59:23.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20824 2.18 MB/sec execute 329 sec latency 4479.856 ms 2024-03-14T21:59:24.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 20983 2.21 MB/sec execute 330 sec latency 5011.810 ms 2024-03-14T21:59:25.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21364 2.25 MB/sec execute 331 sec latency 95.402 ms 2024-03-14T21:59:26.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21448 2.26 MB/sec execute 332 sec latency 825.241 ms 2024-03-14T21:59:27.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21450 2.25 MB/sec execute 333 sec latency 1825.310 ms 2024-03-14T21:59:28.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21461 2.24 MB/sec execute 334 sec latency 2189.394 ms 2024-03-14T21:59:29.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21467 2.24 MB/sec execute 335 sec latency 1630.149 ms 2024-03-14T21:59:30.560 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21470 2.23 MB/sec execute 336 sec latency 2272.348 ms 2024-03-14T21:59:31.570 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21476 2.23 MB/sec execute 337 sec latency 3282.309 ms 2024-03-14T21:59:32.570 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21478 2.22 MB/sec execute 338 sec latency 3631.317 ms 2024-03-14T21:59:33.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21481 2.21 MB/sec execute 339 sec latency 2017.138 ms 2024-03-14T21:59:34.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21485 2.21 MB/sec execute 340 sec latency 2292.196 ms 2024-03-14T21:59:35.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21487 2.20 MB/sec execute 341 sec latency 2568.429 ms 2024-03-14T21:59:36.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21490 2.19 MB/sec execute 342 sec latency 3568.399 ms 2024-03-14T21:59:37.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21497 2.19 MB/sec execute 343 sec latency 3622.685 ms 2024-03-14T21:59:38.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21500 2.18 MB/sec execute 344 sec latency 1814.994 ms 2024-03-14T21:59:39.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21510 2.18 MB/sec execute 345 sec latency 2179.113 ms 2024-03-14T21:59:40.572 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21516 2.17 MB/sec execute 346 sec latency 1401.968 ms 2024-03-14T21:59:41.574 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21518 2.17 MB/sec execute 347 sec latency 2403.438 ms 2024-03-14T21:59:42.574 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21532 2.16 MB/sec execute 348 sec latency 2919.005 ms 2024-03-14T21:59:43.624 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21532 2.15 MB/sec execute 349 sec latency 1900.400 ms 2024-03-14T21:59:44.624 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21532 2.15 MB/sec execute 350 sec latency 2900.430 ms 2024-03-14T21:59:45.624 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21532 2.14 MB/sec execute 351 sec latency 3900.458 ms 2024-03-14T21:59:46.624 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21532 2.14 MB/sec execute 352 sec latency 4900.487 ms 2024-03-14T21:59:46.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:59:46.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:59:46.829 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T21:59:47.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21539 2.13 MB/sec execute 353 sec latency 5090.673 ms 2024-03-14T21:59:48.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21540 2.12 MB/sec execute 354 sec latency 5103.036 ms 2024-03-14T21:59:49.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21543 2.12 MB/sec execute 355 sec latency 2190.748 ms 2024-03-14T21:59:50.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21547 2.11 MB/sec execute 356 sec latency 1851.120 ms 2024-03-14T21:59:51.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21550 2.11 MB/sec execute 357 sec latency 1866.167 ms 2024-03-14T21:59:52.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21553 2.10 MB/sec execute 358 sec latency 2590.650 ms 2024-03-14T21:59:53.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21576 2.10 MB/sec execute 359 sec latency 2221.014 ms 2024-03-14T21:59:54.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21582 2.10 MB/sec execute 360 sec latency 2275.220 ms 2024-03-14T21:59:55.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21587 2.10 MB/sec execute 361 sec latency 3275.255 ms 2024-03-14T21:59:56.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21587 2.09 MB/sec execute 362 sec latency 4275.283 ms 2024-03-14T21:59:57.625 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21597 2.08 MB/sec execute 363 sec latency 4742.865 ms 2024-03-14T21:59:58.626 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21605 2.08 MB/sec execute 364 sec latency 1469.123 ms 2024-03-14T21:59:59.626 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21605 2.07 MB/sec execute 365 sec latency 1793.310 ms 2024-03-14T22:00:00.626 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21606 2.07 MB/sec execute 366 sec latency 2793.386 ms 2024-03-14T22:00:01.626 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21616 2.06 MB/sec execute 367 sec latency 3062.516 ms 2024-03-14T22:00:02.626 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21619 2.06 MB/sec execute 368 sec latency 3630.102 ms 2024-03-14T22:00:03.626 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21625 2.05 MB/sec execute 369 sec latency 2901.265 ms 2024-03-14T22:00:04.626 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21626 2.05 MB/sec execute 370 sec latency 3882.241 ms 2024-03-14T22:00:05.628 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21629 2.04 MB/sec execute 371 sec latency 3988.684 ms 2024-03-14T22:00:06.640 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21632 2.04 MB/sec execute 372 sec latency 2550.198 ms 2024-03-14T22:00:07.640 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21632 2.03 MB/sec execute 373 sec latency 2907.327 ms 2024-03-14T22:00:08.640 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21637 2.02 MB/sec execute 374 sec latency 2913.047 ms 2024-03-14T22:00:09.643 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21642 2.02 MB/sec execute 375 sec latency 2538.318 ms 2024-03-14T22:00:10.643 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21651 2.01 MB/sec execute 376 sec latency 2549.327 ms 2024-03-14T22:00:11.643 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21656 2.01 MB/sec execute 377 sec latency 1465.695 ms 2024-03-14T22:00:12.643 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21661 2.01 MB/sec execute 378 sec latency 2077.037 ms 2024-03-14T22:00:13.643 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21670 2.00 MB/sec execute 379 sec latency 2171.537 ms 2024-03-14T22:00:14.653 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21674 2.00 MB/sec execute 380 sec latency 1552.225 ms 2024-03-14T22:00:15.653 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21687 2.00 MB/sec execute 381 sec latency 2547.465 ms 2024-03-14T22:00:16.653 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21688 2.00 MB/sec execute 382 sec latency 1728.662 ms 2024-03-14T22:00:17.020 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:00:17.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:00:17.026 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:00:17.653 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21688 1.99 MB/sec execute 383 sec latency 2728.731 ms 2024-03-14T22:00:18.653 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21688 1.99 MB/sec execute 384 sec latency 3728.787 ms 2024-03-14T22:00:19.661 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21691 1.98 MB/sec execute 385 sec latency 4736.907 ms 2024-03-14T22:00:20.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21697 1.98 MB/sec execute 386 sec latency 5384.769 ms 2024-03-14T22:00:21.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21708 1.97 MB/sec execute 387 sec latency 5383.548 ms 2024-03-14T22:00:22.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21714 1.97 MB/sec execute 388 sec latency 1219.962 ms 2024-03-14T22:00:23.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21716 1.96 MB/sec execute 389 sec latency 2162.662 ms 2024-03-14T22:00:24.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21718 1.96 MB/sec execute 390 sec latency 2571.125 ms 2024-03-14T22:00:25.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21733 1.96 MB/sec execute 391 sec latency 1473.852 ms 2024-03-14T22:00:26.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21737 1.95 MB/sec execute 392 sec latency 1455.817 ms 2024-03-14T22:00:27.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21743 1.95 MB/sec execute 393 sec latency 2178.631 ms 2024-03-14T22:00:28.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21749 1.94 MB/sec execute 394 sec latency 3178.688 ms 2024-03-14T22:00:29.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21759 1.94 MB/sec execute 395 sec latency 4178.741 ms 2024-03-14T22:00:30.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21766 1.94 MB/sec execute 396 sec latency 4718.919 ms 2024-03-14T22:00:31.662 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21773 1.93 MB/sec execute 397 sec latency 2906.789 ms 2024-03-14T22:00:32.666 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21779 1.93 MB/sec execute 398 sec latency 2176.639 ms 2024-03-14T22:00:33.666 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21781 1.92 MB/sec execute 399 sec latency 2374.469 ms 2024-03-14T22:00:34.666 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21787 1.92 MB/sec execute 400 sec latency 2546.217 ms 2024-03-14T22:00:35.667 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21788 1.91 MB/sec execute 401 sec latency 3285.124 ms 2024-03-14T22:00:36.670 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21789 1.91 MB/sec execute 402 sec latency 4004.016 ms 2024-03-14T22:00:37.670 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21796 1.91 MB/sec execute 403 sec latency 4003.036 ms 2024-03-14T22:00:38.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21799 1.90 MB/sec execute 404 sec latency 3636.677 ms 2024-03-14T22:00:39.671 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21799 1.90 MB/sec execute 405 sec latency 2922.782 ms 2024-03-14T22:00:40.671 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21799 1.89 MB/sec execute 406 sec latency 3922.815 ms 2024-03-14T22:00:41.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21802 1.89 MB/sec execute 407 sec latency 4726.441 ms 2024-03-14T22:00:42.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21808 1.88 MB/sec execute 408 sec latency 3638.113 ms 2024-03-14T22:00:43.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21824 1.89 MB/sec execute 409 sec latency 2175.486 ms 2024-03-14T22:00:44.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21826 1.88 MB/sec execute 410 sec latency 1748.440 ms 2024-03-14T22:00:45.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21828 1.88 MB/sec execute 411 sec latency 2176.834 ms 2024-03-14T22:00:46.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21831 1.87 MB/sec execute 412 sec latency 2660.210 ms 2024-03-14T22:00:47.177 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:00:47.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:00:47.182 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:00:47.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21834 1.87 MB/sec execute 413 sec latency 3271.387 ms 2024-03-14T22:00:48.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21836 1.87 MB/sec execute 414 sec latency 2117.580 ms 2024-03-14T22:00:49.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21840 1.86 MB/sec execute 415 sec latency 2177.665 ms 2024-03-14T22:00:50.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21848 1.86 MB/sec execute 416 sec latency 2542.199 ms 2024-03-14T22:00:51.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21860 1.85 MB/sec execute 417 sec latency 2215.331 ms 2024-03-14T22:00:52.674 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21863 1.85 MB/sec execute 418 sec latency 2914.175 ms 2024-03-14T22:00:53.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21866 1.84 MB/sec execute 419 sec latency 2180.851 ms 2024-03-14T22:00:54.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21873 1.84 MB/sec execute 420 sec latency 1573.325 ms 2024-03-14T22:00:55.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21880 1.84 MB/sec execute 421 sec latency 2177.542 ms 2024-03-14T22:00:56.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 21883 1.83 MB/sec execute 422 sec latency 2484.847 ms 2024-03-14T22:00:57.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22179 1.86 MB/sec execute 423 sec latency 2586.536 ms 2024-03-14T22:00:58.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22312 1.87 MB/sec execute 424 sec latency 779.318 ms 2024-03-14T22:00:59.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22312 1.87 MB/sec execute 425 sec latency 1779.410 ms 2024-03-14T22:01:00.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22312 1.86 MB/sec execute 426 sec latency 2779.489 ms 2024-03-14T22:01:01.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22312 1.86 MB/sec execute 427 sec latency 3779.545 ms 2024-03-14T22:01:02.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22312 1.85 MB/sec execute 428 sec latency 4779.605 ms 2024-03-14T22:01:03.675 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22313 1.85 MB/sec execute 429 sec latency 5779.658 ms 2024-03-14T22:01:04.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22384 1.85 MB/sec execute 430 sec latency 6652.271 ms 2024-03-14T22:01:05.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 22830 1.89 MB/sec execute 431 sec latency 128.778 ms 2024-03-14T22:01:06.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 23237 1.95 MB/sec execute 432 sec latency 118.075 ms 2024-03-14T22:01:07.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 23488 1.99 MB/sec execute 433 sec latency 448.973 ms 2024-03-14T22:01:08.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 23942 2.05 MB/sec execute 434 sec latency 99.649 ms 2024-03-14T22:01:09.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24142 2.06 MB/sec execute 435 sec latency 488.812 ms 2024-03-14T22:01:10.676 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24142 2.06 MB/sec execute 436 sec latency 1488.900 ms 2024-03-14T22:01:11.677 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24142 2.05 MB/sec execute 437 sec latency 2489.013 ms 2024-03-14T22:01:12.677 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24142 2.05 MB/sec execute 438 sec latency 3489.098 ms 2024-03-14T22:01:13.677 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24142 2.04 MB/sec execute 439 sec latency 4489.176 ms 2024-03-14T22:01:14.677 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24293 2.05 MB/sec execute 440 sec latency 4987.089 ms 2024-03-14T22:01:15.680 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24486 2.06 MB/sec execute 441 sec latency 384.310 ms 2024-03-14T22:01:16.680 INFO:tasks.workunit.client.0.smithi163.stdout: 10 24794 2.10 MB/sec execute 442 sec latency 389.314 ms 2024-03-14T22:01:17.668 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:01:17.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:01:17.673 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:01:17.682 INFO:tasks.workunit.client.0.smithi163.stdout: 10 25089 2.13 MB/sec execute 443 sec latency 215.085 ms 2024-03-14T22:01:18.681 INFO:tasks.workunit.client.0.smithi163.stdout: 10 25532 2.19 MB/sec execute 444 sec latency 133.364 ms 2024-03-14T22:01:19.681 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26002 2.23 MB/sec execute 445 sec latency 108.999 ms 2024-03-14T22:01:20.681 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26112 2.23 MB/sec execute 446 sec latency 771.687 ms 2024-03-14T22:01:21.681 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26112 2.22 MB/sec execute 447 sec latency 1771.768 ms 2024-03-14T22:01:22.681 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26112 2.22 MB/sec execute 448 sec latency 2771.849 ms 2024-03-14T22:01:23.681 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26112 2.21 MB/sec execute 449 sec latency 3771.928 ms 2024-03-14T22:01:24.681 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26112 2.21 MB/sec execute 450 sec latency 4772.006 ms 2024-03-14T22:01:25.682 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26383 2.24 MB/sec execute 451 sec latency 5034.666 ms 2024-03-14T22:01:26.682 INFO:tasks.workunit.client.0.smithi163.stdout: 10 26813 2.31 MB/sec execute 452 sec latency 221.422 ms 2024-03-14T22:01:27.682 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27195 2.36 MB/sec execute 453 sec latency 136.400 ms 2024-03-14T22:01:28.682 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27553 2.38 MB/sec execute 454 sec latency 98.530 ms 2024-03-14T22:01:29.682 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27838 2.40 MB/sec execute 455 sec latency 163.375 ms 2024-03-14T22:01:30.682 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27979 2.41 MB/sec execute 456 sec latency 542.975 ms 2024-03-14T22:01:31.683 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27979 2.40 MB/sec execute 457 sec latency 1543.060 ms 2024-03-14T22:01:32.683 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27979 2.40 MB/sec execute 458 sec latency 2543.146 ms 2024-03-14T22:01:33.683 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27979 2.39 MB/sec execute 459 sec latency 3543.221 ms 2024-03-14T22:01:34.692 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27979 2.39 MB/sec execute 460 sec latency 4552.248 ms 2024-03-14T22:01:35.692 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27979 2.38 MB/sec execute 461 sec latency 5552.309 ms 2024-03-14T22:01:36.692 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27987 2.38 MB/sec execute 462 sec latency 6198.601 ms 2024-03-14T22:01:37.693 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27995 2.37 MB/sec execute 463 sec latency 1093.189 ms 2024-03-14T22:01:38.693 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27998 2.37 MB/sec execute 464 sec latency 1539.287 ms 2024-03-14T22:01:39.693 INFO:tasks.workunit.client.0.smithi163.stdout: 10 27999 2.36 MB/sec execute 465 sec latency 2539.359 ms 2024-03-14T22:01:40.693 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28003 2.36 MB/sec execute 466 sec latency 3539.407 ms 2024-03-14T22:01:41.697 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28005 2.35 MB/sec execute 467 sec latency 4543.293 ms 2024-03-14T22:01:42.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28024 2.35 MB/sec execute 468 sec latency 4720.549 ms 2024-03-14T22:01:43.705 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28028 2.35 MB/sec execute 469 sec latency 1457.669 ms 2024-03-14T22:01:44.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28029 2.34 MB/sec execute 470 sec latency 2180.386 ms 2024-03-14T22:01:45.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28029 2.34 MB/sec execute 471 sec latency 3096.244 ms 2024-03-14T22:01:46.708 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28037 2.33 MB/sec execute 472 sec latency 3624.876 ms 2024-03-14T22:01:47.716 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28038 2.33 MB/sec execute 473 sec latency 1456.286 ms 2024-03-14T22:01:47.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:01:47.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:01:47.826 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:01:47.857 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-03-14T22:01:47.863+0000 7f5aad84b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 47124) UID: 0 2024-03-14T22:01:47.858 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-03-14T22:01:47.863+0000 7f137ed27640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 47124) UID: 0 2024-03-14T22:01:48.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28039 2.32 MB/sec execute 474 sec latency 2116.133 ms 2024-03-14T22:01:49.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28042 2.32 MB/sec execute 475 sec latency 3116.166 ms 2024-03-14T22:01:50.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28046 2.31 MB/sec execute 476 sec latency 4116.201 ms 2024-03-14T22:01:51.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28051 2.31 MB/sec execute 477 sec latency 5116.268 ms 2024-03-14T22:01:52.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28057 2.30 MB/sec execute 478 sec latency 6116.319 ms 2024-03-14T22:01:53.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28057 2.30 MB/sec execute 479 sec latency 7116.354 ms 2024-03-14T22:01:54.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28057 2.29 MB/sec execute 480 sec latency 8116.392 ms 2024-03-14T22:01:55.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28057 2.29 MB/sec execute 481 sec latency 9116.448 ms 2024-03-14T22:01:56.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28057 2.28 MB/sec execute 482 sec latency 10116.501 ms 2024-03-14T22:01:57.717 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28059 2.28 MB/sec execute 483 sec latency 10902.879 ms 2024-03-14T22:01:58.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28072 2.28 MB/sec execute 484 sec latency 6203.639 ms 2024-03-14T22:01:59.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28080 2.28 MB/sec execute 485 sec latency 2208.837 ms 2024-03-14T22:02:00.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28094 2.28 MB/sec execute 486 sec latency 1004.980 ms 2024-03-14T22:02:01.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28097 2.27 MB/sec execute 487 sec latency 2005.021 ms 2024-03-14T22:02:02.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28101 2.27 MB/sec execute 488 sec latency 2558.335 ms 2024-03-14T22:02:03.718 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28105 2.26 MB/sec execute 489 sec latency 2162.970 ms 2024-03-14T22:02:04.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28105 2.26 MB/sec execute 490 sec latency 3164.979 ms 2024-03-14T22:02:05.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28105 2.25 MB/sec execute 491 sec latency 4165.020 ms 2024-03-14T22:02:06.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28105 2.25 MB/sec execute 492 sec latency 5165.094 ms 2024-03-14T22:02:07.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28105 2.25 MB/sec execute 493 sec latency 6165.148 ms 2024-03-14T22:02:08.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28114 2.25 MB/sec execute 494 sec latency 6571.531 ms 2024-03-14T22:02:09.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28116 2.24 MB/sec execute 495 sec latency 1593.825 ms 2024-03-14T22:02:10.720 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28123 2.24 MB/sec execute 496 sec latency 2176.877 ms 2024-03-14T22:02:11.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28123 2.23 MB/sec execute 497 sec latency 1415.952 ms 2024-03-14T22:02:12.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28126 2.23 MB/sec execute 498 sec latency 2416.012 ms 2024-03-14T22:02:13.721 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28128 2.22 MB/sec execute 499 sec latency 2541.999 ms 2024-03-14T22:02:14.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28135 2.22 MB/sec execute 500 sec latency 2621.725 ms 2024-03-14T22:02:15.743 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28143 2.22 MB/sec execute 501 sec latency 2902.830 ms 2024-03-14T22:02:16.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28144 2.21 MB/sec execute 502 sec latency 1720.107 ms 2024-03-14T22:02:17.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28155 2.21 MB/sec execute 503 sec latency 2539.359 ms 2024-03-14T22:02:18.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28161 2.20 MB/sec execute 504 sec latency 1455.751 ms 2024-03-14T22:02:19.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28166 2.20 MB/sec execute 505 sec latency 1820.047 ms 2024-03-14T22:02:20.745 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28175 2.20 MB/sec execute 506 sec latency 1360.642 ms 2024-03-14T22:02:21.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28178 2.19 MB/sec execute 507 sec latency 2185.295 ms 2024-03-14T22:02:22.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28182 2.19 MB/sec execute 508 sec latency 2630.800 ms 2024-03-14T22:02:23.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28188 2.18 MB/sec execute 509 sec latency 2916.127 ms 2024-03-14T22:02:24.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28199 2.18 MB/sec execute 510 sec latency 2564.379 ms 2024-03-14T22:02:25.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28205 2.18 MB/sec execute 511 sec latency 1610.822 ms 2024-03-14T22:02:26.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28205 2.17 MB/sec execute 512 sec latency 2610.885 ms 2024-03-14T22:02:27.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28205 2.17 MB/sec execute 513 sec latency 3610.922 ms 2024-03-14T22:02:28.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28205 2.16 MB/sec execute 514 sec latency 4610.982 ms 2024-03-14T22:02:29.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28205 2.16 MB/sec execute 515 sec latency 5611.045 ms 2024-03-14T22:02:30.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28210 2.15 MB/sec execute 516 sec latency 6217.147 ms 2024-03-14T22:02:31.746 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28216 2.15 MB/sec execute 517 sec latency 759.135 ms 2024-03-14T22:02:32.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28226 2.15 MB/sec execute 518 sec latency 1459.378 ms 2024-03-14T22:02:33.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28230 2.15 MB/sec execute 519 sec latency 1398.294 ms 2024-03-14T22:02:34.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28233 2.14 MB/sec execute 520 sec latency 2398.334 ms 2024-03-14T22:02:35.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28245 2.14 MB/sec execute 521 sec latency 3157.991 ms 2024-03-14T22:02:36.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28252 2.14 MB/sec execute 522 sec latency 3276.133 ms 2024-03-14T22:02:37.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28253 2.13 MB/sec execute 523 sec latency 1881.855 ms 2024-03-14T22:02:38.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28253 2.13 MB/sec execute 524 sec latency 2881.903 ms 2024-03-14T22:02:39.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28253 2.13 MB/sec execute 525 sec latency 3881.937 ms 2024-03-14T22:02:40.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28253 2.12 MB/sec execute 526 sec latency 4881.972 ms 2024-03-14T22:02:41.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28257 2.12 MB/sec execute 527 sec latency 5517.404 ms 2024-03-14T22:02:42.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28261 2.11 MB/sec execute 528 sec latency 6179.921 ms 2024-03-14T22:02:43.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28264 2.11 MB/sec execute 529 sec latency 2431.954 ms 2024-03-14T22:02:44.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28265 2.11 MB/sec execute 530 sec latency 2910.903 ms 2024-03-14T22:02:45.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28269 2.10 MB/sec execute 531 sec latency 2917.433 ms 2024-03-14T22:02:46.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28273 2.10 MB/sec execute 532 sec latency 2551.440 ms 2024-03-14T22:02:47.747 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28273 2.09 MB/sec execute 533 sec latency 2425.976 ms 2024-03-14T22:02:48.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28273 2.09 MB/sec execute 534 sec latency 3426.032 ms 2024-03-14T22:02:49.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28273 2.09 MB/sec execute 535 sec latency 4426.089 ms 2024-03-14T22:02:50.748 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28273 2.08 MB/sec execute 536 sec latency 5426.146 ms 2024-03-14T22:02:51.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28273 2.08 MB/sec execute 537 sec latency 6429.003 ms 2024-03-14T22:02:52.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28279 2.08 MB/sec execute 538 sec latency 6901.864 ms 2024-03-14T22:02:53.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28283 2.07 MB/sec execute 539 sec latency 7266.119 ms 2024-03-14T22:02:54.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28286 2.07 MB/sec execute 540 sec latency 2161.874 ms 2024-03-14T22:02:55.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28312 2.07 MB/sec execute 541 sec latency 2181.768 ms 2024-03-14T22:02:56.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28316 2.06 MB/sec execute 542 sec latency 1816.071 ms 2024-03-14T22:02:57.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28359 2.06 MB/sec execute 543 sec latency 2178.117 ms 2024-03-14T22:02:58.751 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28508 2.08 MB/sec execute 544 sec latency 539.638 ms 2024-03-14T22:02:59.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28508 2.07 MB/sec execute 545 sec latency 1539.764 ms 2024-03-14T22:03:00.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28508 2.07 MB/sec execute 546 sec latency 2539.869 ms 2024-03-14T22:03:01.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28508 2.06 MB/sec execute 547 sec latency 3539.973 ms 2024-03-14T22:03:02.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28508 2.06 MB/sec execute 548 sec latency 4540.087 ms 2024-03-14T22:03:03.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28511 2.06 MB/sec execute 549 sec latency 5359.314 ms 2024-03-14T22:03:04.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 28571 2.06 MB/sec execute 550 sec latency 730.884 ms 2024-03-14T22:03:05.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 29106 2.10 MB/sec execute 551 sec latency 104.600 ms 2024-03-14T22:03:06.752 INFO:tasks.workunit.client.0.smithi163.stdout: 10 29488 2.13 MB/sec execute 552 sec latency 142.193 ms 2024-03-14T22:03:07.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 29830 2.17 MB/sec execute 553 sec latency 177.691 ms 2024-03-14T22:03:08.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30143 2.20 MB/sec execute 554 sec latency 176.936 ms 2024-03-14T22:03:09.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30213 2.20 MB/sec execute 555 sec latency 814.536 ms 2024-03-14T22:03:10.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30213 2.20 MB/sec execute 556 sec latency 1814.578 ms 2024-03-14T22:03:11.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30213 2.20 MB/sec execute 557 sec latency 2814.607 ms 2024-03-14T22:03:12.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30213 2.19 MB/sec execute 558 sec latency 3814.634 ms 2024-03-14T22:03:13.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30213 2.19 MB/sec execute 559 sec latency 4814.662 ms 2024-03-14T22:03:14.753 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30456 2.21 MB/sec execute 560 sec latency 5057.344 ms 2024-03-14T22:03:15.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30608 2.22 MB/sec execute 561 sec latency 642.546 ms 2024-03-14T22:03:16.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 30919 2.24 MB/sec execute 562 sec latency 754.035 ms 2024-03-14T22:03:17.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31221 2.26 MB/sec execute 563 sec latency 176.353 ms 2024-03-14T22:03:18.757 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31487 2.27 MB/sec execute 564 sec latency 246.212 ms 2024-03-14T22:03:19.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31668 2.30 MB/sec execute 565 sec latency 543.466 ms 2024-03-14T22:03:20.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31668 2.29 MB/sec execute 566 sec latency 1543.586 ms 2024-03-14T22:03:21.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31668 2.29 MB/sec execute 567 sec latency 2543.707 ms 2024-03-14T22:03:22.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31668 2.28 MB/sec execute 568 sec latency 3543.814 ms 2024-03-14T22:03:23.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31668 2.28 MB/sec execute 569 sec latency 4543.923 ms 2024-03-14T22:03:24.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 31945 2.30 MB/sec execute 570 sec latency 5016.968 ms 2024-03-14T22:03:25.758 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32503 2.35 MB/sec execute 571 sec latency 91.795 ms 2024-03-14T22:03:26.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32852 2.37 MB/sec execute 572 sec latency 409.755 ms 2024-03-14T22:03:27.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32857 2.37 MB/sec execute 573 sec latency 1375.252 ms 2024-03-14T22:03:28.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32857 2.36 MB/sec execute 574 sec latency 2375.306 ms 2024-03-14T22:03:29.759 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32871 2.36 MB/sec execute 575 sec latency 3375.366 ms 2024-03-14T22:03:30.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32875 2.36 MB/sec execute 576 sec latency 4001.881 ms 2024-03-14T22:03:31.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32877 2.35 MB/sec execute 577 sec latency 1005.787 ms 2024-03-14T22:03:32.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32877 2.35 MB/sec execute 578 sec latency 2005.642 ms 2024-03-14T22:03:33.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32883 2.35 MB/sec execute 579 sec latency 2638.621 ms 2024-03-14T22:03:34.761 INFO:tasks.workunit.client.0.smithi163.stdout: 10 32883 2.34 MB/sec execute 580 sec latency 3638.711 ms 2024-03-14T22:03:35.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 33016 2.35 MB/sec execute 581 sec latency 4267.002 ms 2024-03-14T22:03:36.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 33444 2.40 MB/sec execute 582 sec latency 132.350 ms 2024-03-14T22:03:37.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 33921 2.46 MB/sec execute 583 sec latency 111.918 ms 2024-03-14T22:03:38.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34073 2.47 MB/sec execute 584 sec latency 614.214 ms 2024-03-14T22:03:39.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34082 2.46 MB/sec execute 585 sec latency 753.528 ms 2024-03-14T22:03:40.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34087 2.46 MB/sec execute 586 sec latency 1491.335 ms 2024-03-14T22:03:41.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34090 2.45 MB/sec execute 587 sec latency 1821.449 ms 2024-03-14T22:03:42.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34091 2.45 MB/sec execute 588 sec latency 2400.055 ms 2024-03-14T22:03:43.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34093 2.45 MB/sec execute 589 sec latency 2539.937 ms 2024-03-14T22:03:44.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34103 2.44 MB/sec execute 590 sec latency 2896.704 ms 2024-03-14T22:03:45.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34107 2.44 MB/sec execute 591 sec latency 1048.593 ms 2024-03-14T22:03:46.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34107 2.43 MB/sec execute 592 sec latency 2048.631 ms 2024-03-14T22:03:47.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34111 2.43 MB/sec execute 593 sec latency 2902.674 ms 2024-03-14T22:03:48.762 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34113 2.43 MB/sec execute 594 sec latency 2542.300 ms 2024-03-14T22:03:49.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34113 2.42 MB/sec execute 595 sec latency 2505.664 ms 2024-03-14T22:03:50.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34121 2.42 MB/sec execute 596 sec latency 2542.418 ms 2024-03-14T22:03:51.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34123 2.41 MB/sec execute 597 sec latency 1596.255 ms 2024-03-14T22:03:52.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34125 2.41 MB/sec execute 598 sec latency 2596.289 ms 2024-03-14T22:03:53.765 INFO:tasks.workunit.client.0.smithi163.stdout: 10 34125 2.41 MB/sec execute 599 sec latency 3596.340 ms 2024-03-14T22:03:54.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 600 sec 2024-03-14T22:03:55.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 601 sec 2024-03-14T22:03:56.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 602 sec 2024-03-14T22:03:57.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 603 sec 2024-03-14T22:03:58.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 604 sec 2024-03-14T22:03:59.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 605 sec 2024-03-14T22:04:00.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 606 sec 2024-03-14T22:04:01.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 607 sec 2024-03-14T22:04:02.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 608 sec 2024-03-14T22:04:03.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 609 sec 2024-03-14T22:04:04.763 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 610 sec 2024-03-14T22:04:05.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 611 sec 2024-03-14T22:04:06.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 612 sec 2024-03-14T22:04:07.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 613 sec 2024-03-14T22:04:08.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 614 sec 2024-03-14T22:04:09.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 615 sec 2024-03-14T22:04:10.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 616 sec 2024-03-14T22:04:11.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 617 sec 2024-03-14T22:04:12.767 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 618 sec 2024-03-14T22:04:13.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 619 sec 2024-03-14T22:04:14.764 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 620 sec 2024-03-14T22:04:15.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 621 sec 2024-03-14T22:04:16.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 622 sec 2024-03-14T22:04:17.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 623 sec 2024-03-14T22:04:18.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 624 sec 2024-03-14T22:04:19.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 625 sec 2024-03-14T22:04:20.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 626 sec 2024-03-14T22:04:21.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 627 sec 2024-03-14T22:04:22.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 628 sec 2024-03-14T22:04:23.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:04:23.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:04:23.344 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:04:23.769 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 629 sec 2024-03-14T22:04:24.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 630 sec 2024-03-14T22:04:25.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 631 sec 2024-03-14T22:04:26.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 632 sec 2024-03-14T22:04:27.770 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 633 sec 2024-03-14T22:04:28.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 634 sec 2024-03-14T22:04:29.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 635 sec 2024-03-14T22:04:30.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 636 sec 2024-03-14T22:04:31.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 637 sec 2024-03-14T22:04:32.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 638 sec 2024-03-14T22:04:33.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 639 sec 2024-03-14T22:04:34.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 640 sec 2024-03-14T22:04:35.823 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 641 sec 2024-03-14T22:04:36.824 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 642 sec 2024-03-14T22:04:37.825 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 643 sec 2024-03-14T22:04:38.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 644 sec 2024-03-14T22:04:39.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 645 sec 2024-03-14T22:04:40.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 646 sec 2024-03-14T22:04:41.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 647 sec 2024-03-14T22:04:42.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 648 sec 2024-03-14T22:04:43.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 649 sec 2024-03-14T22:04:44.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 650 sec 2024-03-14T22:04:45.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 651 sec 2024-03-14T22:04:46.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 652 sec 2024-03-14T22:04:47.826 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 653 sec 2024-03-14T22:04:48.864 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 655 sec 2024-03-14T22:04:49.864 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 656 sec 2024-03-14T22:04:50.865 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 657 sec 2024-03-14T22:04:51.864 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 658 sec 2024-03-14T22:04:52.864 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 659 sec 2024-03-14T22:04:53.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:04:53.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:04:53.490 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:04:53.865 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 660 sec 2024-03-14T22:04:54.865 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 661 sec 2024-03-14T22:04:55.865 INFO:tasks.workunit.client.0.smithi163.stdout: 10 cleanup 662 sec 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: 0 cleanup 662 sec 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: Operation Count AvgLat MaxLat 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: ---------------------------------------- 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: NTCreateX 46270 46.224 19223.788 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: Close 34085 0.020 361.903 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: Rename 1968 236.980 10902.873 2024-03-14T22:04:56.344 INFO:tasks.workunit.client.0.smithi163.stdout: Unlink 9271 99.734 8343.935 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: Qpathinfo 41998 35.225 9070.489 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: Qfileinfo 7334 0.005 0.215 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: Qfsinfo 7668 16.174 5457.637 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: Sfileinfo 3787 30.962 6725.616 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: Find 16203 11.532 7263.081 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: WriteX 22890 6.723 7266.111 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: ReadX 72728 1.119 6533.484 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: LockX 150 70.761 5088.098 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: UnlockX 150 51.268 1451.394 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: Flush 3237 107.333 6901.857 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout: 2024-03-14T22:04:56.345 INFO:tasks.workunit.client.0.smithi163.stdout:Throughput 2.40726 MB/sec 10 clients 10 procs max_latency=19223.797 ms 2024-03-14T22:04:56.346 INFO:tasks.workunit.client.0.smithi163.stderr:+ popd 2024-03-14T22:04:56.346 INFO:tasks.workunit.client.0.smithi163.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-14T22:04:56.346 INFO:tasks.workunit.client.0.smithi163.stderr:+ clean_up 2024-03-14T22:04:56.346 INFO:tasks.workunit.client.0.smithi163.stderr:+ rm -rf xfprogs-dev-dir 2024-03-14T22:04:56.349 INFO:tasks.workunit.client.0.smithi163.stderr:+ rm -rf xfstest-dev-dir 2024-03-14T22:05:16.656 INFO:tasks.workunit.client.0.smithi163.stderr:+ rm -rf fscrypt_test_unlocked_dbench 2024-03-14T22:05:17.274 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-14T22:05:17.275 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-14T22:05:17.384 INFO:tasks.workunit:Stopping ['fs/fscrypt.sh none dbench', 'fs/fscrypt.sh unlocked dbench'] on client.0... 2024-03-14T22:05:17.385 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-14T22:05:18.374 DEBUG:teuthology.parallel:result is None 2024-03-14T22:05:18.374 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-14T22:05:23.023 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-14T22:05:23.024 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-14T22:05:23.037 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-14T22:05:23.037 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T22:05:23.037 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T22:05:23.079 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-14T22:05:23.079 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T22:05:23.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T22:05:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.490+0000 7f8ff9d3e640 1 Processor -- start 2024-03-14T22:05:23.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.490+0000 7f8ff9d3e640 1 -- start start 2024-03-14T22:05:23.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff9d3e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff40b7e80 0x7f8ff4071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff9d3e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ff40bc340 0x7f8ff40badc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff9d3e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff4072310 con 0x7f8ff40bbed0 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff9d3e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff40b86a0 con 0x7f8ff40bc340 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff9d3e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff409bf60 con 0x7f8ff40b7e80 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ff40bc340 0x7f8ff40badc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ff40bc340 0x7f8ff40badc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50026/0 (socket says 172.21.15.49:50026) 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff3fff640 1 -- 172.21.15.49:0/1247433810 learned_addr learned my addr 172.21.15.49:0/1247433810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff37fe640 1 --2- 172.21.15.49:0/1247433810 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff40b7e80 0x7f8ff4071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff3fff640 1 -- 172.21.15.49:0/1247433810 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff40b7e80 msgr2=0x7f8ff4071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff3fff640 1 --2- 172.21.15.49:0/1247433810 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff40b7e80 0x7f8ff4071dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff3fff640 1 -- 172.21.15.49:0/1247433810 >> v1:172.21.15.97:6789/0 conn(0x7f8ff40bbed0 legacy=0x7f8ff40ba6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.491+0000 7f8ff3fff640 1 -- 172.21.15.49:0/1247433810 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff409c240 con 0x7f8ff40bc340 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff3fff640 1 --2- 172.21.15.49:0/1247433810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40bc340 0x7f8ff40badc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009e50 tx=0x7f8fec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0af04d0821e1d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff27fc640 1 -- 172.21.15.49:0/1247433810 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec02ca70 con 0x7f8ff40bc340 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff27fc640 1 -- 172.21.15.49:0/1247433810 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec02cbd0 con 0x7f8ff40bc340 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff27fc640 1 -- 172.21.15.49:0/1247433810 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0343f0 con 0x7f8ff40bc340 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/1247433810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40bc340 msgr2=0x7f8ff40badc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/1247433810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40bc340 0x7f8ff40badc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009e50 tx=0x7f8fec02bed0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/1247433810 shutdown_connections 2024-03-14T22:05:23.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/1247433810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40bc340 0x7f8ff40badc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/1247433810 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff40b7e80 0x7f8ff4071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.492+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/1247433810 >> 172.21.15.49:0/1247433810 conn(0x7f8ff40b93b0 msgr2=0x7f8ff40b97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:23.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.493+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/1247433810 shutdown_connections 2024-03-14T22:05:23.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.493+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/1247433810 wait complete. 2024-03-14T22:05:23.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 Processor -- start 2024-03-14T22:05:23.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 -- start start 2024-03-14T22:05:23.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40b7e80 0x7f8ff4146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff414d130 0x7f8ff414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ff414c410 0x7f8ff41e3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4147900 con 0x7f8ff40b7e80 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff414daa0 con 0x7f8ff414c410 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.494+0000 7f8ff9d3e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff414bb00 con 0x7f8ff414d130 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40b7e80 0x7f8ff4146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40b7e80 0x7f8ff4146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50042/0 (socket says 172.21.15.49:50042) 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 -- 172.21.15.49:0/2360425279 learned_addr learned my addr 172.21.15.49:0/2360425279 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff2ffd640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff414d130 0x7f8ff414bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff3fff640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ff414c410 0x7f8ff41e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 -- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff414d130 msgr2=0x7f8ff414bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff414d130 0x7f8ff414bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 -- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ff414c410 msgr2=0x7f8ff41e3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ff414c410 0x7f8ff41e3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 -- 172.21.15.49:0/2360425279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec009b30 con 0x7f8ff40b7e80 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff2ffd640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff414d130 0x7f8ff414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff37fe640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40b7e80 0x7f8ff4146a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d800 tx=0x7f8fe400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8fdbfff640 1 -- 172.21.15.49:0/2360425279 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400b6c0 con 0x7f8ff40b7e80 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8fdbfff640 1 -- 172.21.15.49:0/2360425279 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400b820 con 0x7f8ff40b7e80 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff4146fb0 con 0x7f8ff40b7e80 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8fdbfff640 1 -- 172.21.15.49:0/2360425279 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400bbb0 con 0x7f8ff40b7e80 2024-03-14T22:05:23.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.495+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff414dcd0 con 0x7f8ff40b7e80 2024-03-14T22:05:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.496+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff414b960 con 0x7f8ff40b7e80 2024-03-14T22:05:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.496+0000 7f8fdbfff640 1 -- 172.21.15.49:0/2360425279 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8fe400bd10 con 0x7f8ff40b7e80 2024-03-14T22:05:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.497+0000 7f8fdbfff640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8fe006a7a0 0x7f8fe006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.497+0000 7f8fdbfff640 1 -- 172.21.15.49:0/2360425279 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8fe4089270 con 0x7f8ff40b7e80 2024-03-14T22:05:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.497+0000 7f8ff2ffd640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8fe006a7a0 0x7f8fe006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.498+0000 7f8ff2ffd640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8fe006a7a0 0x7f8fe006cc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40b8f90 tx=0x7f8fe8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:23.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.502+0000 7f8fdbfff640 1 -- 172.21.15.49:0/2360425279 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8ff414b960 con 0x7f8ff40b7e80 2024-03-14T22:05:23.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:05:23.583 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:05:23.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:05:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.683+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ff414b960 con 0x7f8ff40b7e80 2024-03-14T22:05:23.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.684+0000 7f8fdbfff640 1 -- 172.21.15.49:0/2360425279 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f8ff414b960 con 0x7f8ff40b7e80 2024-03-14T22:05:23.684 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:05:23.684 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:44.734113+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T22:05:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8fe006a7a0 msgr2=0x7f8fe006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8fe006a7a0 0x7f8fe006cc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40b8f90 tx=0x7f8fe8009040 comp rx=0 tx=0).stop 2024-03-14T22:05:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40b7e80 msgr2=0x7f8ff4146a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40b7e80 0x7f8ff4146a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d800 tx=0x7f8fe400dcc0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 shutdown_connections 2024-03-14T22:05:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8fe006a7a0 0x7f8fe006cc60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ff414c410 0x7f8ff41e3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ff414d130 0x7f8ff414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 --2- 172.21.15.49:0/2360425279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ff40b7e80 0x7f8ff4146a70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.686+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 >> 172.21.15.49:0/2360425279 conn(0x7f8ff40b93b0 msgr2=0x7f8ff406bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.687+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 shutdown_connections 2024-03-14T22:05:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.687+0000 7f8ff9d3e640 1 -- 172.21.15.49:0/2360425279 wait complete. 2024-03-14T22:05:23.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T21:13:50.964908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T21:14:01.020268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T21:14:01.995660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T22:05:23.705 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T22:05:23.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.815+0000 7fb66fc96640 1 Processor -- start 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.815+0000 7fb66fc96640 1 -- start start 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66fc96640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66fc96640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6680b8c40 0x7fb66814e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66fc96640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66806e540 con 0x7fb6680b9080 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66fc96640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66814e090 con 0x7fb6680b8c40 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66fc96640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680b9970 con 0x7fb66814b660 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66d231640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6680b8c40 0x7fb66814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66da32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66e233640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb6680b9080 0x7fb668069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:57086/0 (socket says 172.21.15.49:57086) 2024-03-14T22:05:23.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66e233640 1 -- 172.21.15.49:0/3165246407 learned_addr learned my addr 172.21.15.49:0/3165246407 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:23.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66d231640 1 -- 172.21.15.49:0/3165246407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 msgr2=0x7fb66806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:23.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66d231640 1 --2- 172.21.15.49:0/3165246407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66d231640 1 -- 172.21.15.49:0/3165246407 >> v1:172.21.15.97:6789/0 conn(0x7fb6680b9080 legacy=0x7fb668069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.816+0000 7fb66d231640 1 -- 172.21.15.49:0/3165246407 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6680703c0 con 0x7fb6680b8c40 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.817+0000 7fb66d231640 1 --2- 172.21.15.49:0/3165246407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b8c40 0x7fb66814e570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097a0 tx=0x7fb65802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e0330f52efadc70 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.817+0000 7fb66ca30640 1 -- 172.21.15.49:0/3165246407 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658030610 con 0x7fb6680b8c40 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.817+0000 7fb66ca30640 1 -- 172.21.15.49:0/3165246407 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb658030770 con 0x7fb6680b8c40 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.817+0000 7fb66ca30640 1 -- 172.21.15.49:0/3165246407 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658030b00 con 0x7fb6680b8c40 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 -- 172.21.15.49:0/3165246407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b8c40 msgr2=0x7fb66814e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 --2- 172.21.15.49:0/3165246407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b8c40 0x7fb66814e570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097a0 tx=0x7fb65802b960 comp rx=0 tx=0).stop 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 -- 172.21.15.49:0/3165246407 shutdown_connections 2024-03-14T22:05:23.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 --2- 172.21.15.49:0/3165246407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b8c40 0x7fb66814e570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 --2- 172.21.15.49:0/3165246407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 -- 172.21.15.49:0/3165246407 >> 172.21.15.49:0/3165246407 conn(0x7fb6680b9d30 msgr2=0x7fb6680ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 -- 172.21.15.49:0/3165246407 shutdown_connections 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.818+0000 7fb66fc96640 1 -- 172.21.15.49:0/3165246407 wait complete. 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.819+0000 7fb66fc96640 1 Processor -- start 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.819+0000 7fb66fc96640 1 -- start start 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66fc96640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6680b8c40 0x7fb66809ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66fc96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b9080 0x7fb6681d6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66fc96640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66809dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66fc96640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6681d6270 con 0x7fb6680b8c40 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66fc96640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66809f4c0 con 0x7fb6680b9080 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66fc96640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66814dd70 con 0x7fb66814b660 2024-03-14T22:05:23.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6680b8c40 0x7fb66809ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6680b8c40 0x7fb66809ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50056/0 (socket says 172.21.15.49:50056) 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 -- 172.21.15.49:0/1833578333 learned_addr learned my addr 172.21.15.49:0/1833578333 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66d231640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b9080 0x7fb6681d6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66e233640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66809dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 -- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 msgr2=0x7fb66809dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66809dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 -- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b9080 msgr2=0x7fb6681d6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b9080 0x7fb6681d6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 -- 172.21.15.49:0/1833578333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb658009400 con 0x7fb6680b8c40 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb66da32640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6680b8c40 0x7fb66809ee40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb66000efc0 tx=0x7fb66000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.820+0000 7fb65e7fc640 1 -- 172.21.15.49:0/1833578333 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660009b80 con 0x7fb6680b8c40 2024-03-14T22:05:23.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.821+0000 7fb65e7fc640 1 -- 172.21.15.49:0/1833578333 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66000f040 con 0x7fb6680b8c40 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.821+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66809fc80 con 0x7fb6680b8c40 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.821+0000 7fb65e7fc640 1 -- 172.21.15.49:0/1833578333 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600146a0 con 0x7fb6680b8c40 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.821+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66809ce60 con 0x7fb6680b8c40 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.822+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb668066dc0 con 0x7fb6680b8c40 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.822+0000 7fb65e7fc640 1 -- 172.21.15.49:0/1833578333 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb660014800 con 0x7fb6680b8c40 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.823+0000 7fb65e7fc640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb64c06a7d0 0x7fb64c06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.823+0000 7fb65e7fc640 1 -- 172.21.15.49:0/1833578333 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb6600897d0 con 0x7fb6680b8c40 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.823+0000 7fb66d231640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb64c06a7d0 0x7fb64c06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:23.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.823+0000 7fb66d231640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb64c06a7d0 0x7fb64c06cc90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb658036000 tx=0x7fb658008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:23.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:23.827+0000 7fb65e7fc640 1 -- 172.21.15.49:0/1833578333 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb660054c20 con 0x7fb6680b8c40 2024-03-14T22:05:24.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.010+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6681d6060 con 0x7fb6680b8c40 2024-03-14T22:05:24.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.010+0000 7fb65e7fc640 1 -- 172.21.15.49:0/1833578333 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb660058240 con 0x7fb6680b8c40 2024-03-14T22:05:24.009 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb64c06a7d0 msgr2=0x7fb64c06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb64c06a7d0 0x7fb64c06cc90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb658036000 tx=0x7fb658008030 comp rx=0 tx=0).stop 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6680b8c40 msgr2=0x7fb66809ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6680b8c40 0x7fb66809ee40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb66000efc0 tx=0x7fb66000c490 comp rx=0 tx=0).stop 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 shutdown_connections 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb64c06a7d0 0x7fb64c06cc90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb66814b660 0x7fb66809dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6680b9080 0x7fb6681d6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 --2- 172.21.15.49:0/1833578333 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6680b8c40 0x7fb66809ee40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 >> 172.21.15.49:0/1833578333 conn(0x7fb6680b9d30 msgr2=0x7fb66814a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:24.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 shutdown_connections 2024-03-14T22:05:24.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.012+0000 7fb66fc96640 1 -- 172.21.15.49:0/1833578333 wait complete. 2024-03-14T22:05:24.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T22:05:24.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.133+0000 7fc6fa7b1640 1 Processor -- start 2024-03-14T22:05:24.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.133+0000 7fc6fa7b1640 1 -- start start 2024-03-14T22:05:24.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.133+0000 7fc6fa7b1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f406ba30 0x7fc6f40bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.133+0000 7fc6fa7b1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f414b760 con 0x7fc6f40be7a0 2024-03-14T22:05:24.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.133+0000 7fc6fa7b1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f414eca0 con 0x7fc6f40bebc0 2024-03-14T22:05:24.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.133+0000 7fc6fa7b1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40be120 con 0x7fc6f406ba30 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f37fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc6f40be7a0 0x7fc6f414de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43540/0 (socket says 172.21.15.49:43540) 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f37fe640 1 -- 172.21.15.49:0/4291039719 learned_addr learned my addr 172.21.15.49:0/4291039719 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f3fff640 1 --2- 172.21.15.49:0/4291039719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f406ba30 0x7fc6f40bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f2ffd640 1 -- 172.21.15.49:0/4291039719 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277218376 0 0) 0x7fc6f414b760 con 0x7fc6f40be7a0 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f2ffd640 1 -- 172.21.15.49:0/4291039719 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6dc003520 con 0x7fc6f40be7a0 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f2ffd640 1 -- 172.21.15.49:0/4291039719 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 723967428 0 0) 0x7fc6dc003520 con 0x7fc6f40be7a0 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f3fff640 1 -- 172.21.15.49:0/4291039719 >> v1:172.21.15.97:6789/0 conn(0x7fc6f40bebc0 legacy=0x7fc6f414e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f3fff640 1 -- 172.21.15.49:0/4291039719 >> v1:172.21.15.49:6789/0 conn(0x7fc6f40be7a0 legacy=0x7fc6f414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.134+0000 7fc6f3fff640 1 -- 172.21.15.49:0/4291039719 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f414a9a0 con 0x7fc6f406ba30 2024-03-14T22:05:24.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.135+0000 7fc6f3fff640 1 --2- 172.21.15.49:0/4291039719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f406ba30 0x7fc6f40bac10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40097d0 tx=0x7fc6e402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bef67569516c90d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.139+0000 7fc6f2ffd640 1 -- 172.21.15.49:0/4291039719 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8001670 con 0x7fc6f406ba30 2024-03-14T22:05:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.139+0000 7fc6f2ffd640 1 -- 172.21.15.49:0/4291039719 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4031580 con 0x7fc6f406ba30 2024-03-14T22:05:24.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.139+0000 7fc6f2ffd640 1 -- 172.21.15.49:0/4291039719 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4031910 con 0x7fc6f406ba30 2024-03-14T22:05:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.139+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/4291039719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f406ba30 msgr2=0x7fc6f40bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.139+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/4291039719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f406ba30 0x7fc6f40bac10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40097d0 tx=0x7fc6e402b8a0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.140+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/4291039719 shutdown_connections 2024-03-14T22:05:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.140+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/4291039719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f406ba30 0x7fc6f40bac10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.140+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/4291039719 >> 172.21.15.49:0/4291039719 conn(0x7fc6f40c0290 msgr2=0x7fc6f40c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.140+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/4291039719 shutdown_connections 2024-03-14T22:05:24.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.140+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/4291039719 wait complete. 2024-03-14T22:05:24.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 Processor -- start 2024-03-14T22:05:24.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 -- start start 2024-03-14T22:05:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6f406ba30 0x7fc6f40a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f40be7a0 0x7fc6f414c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f40bebc0 0x7fc6f41e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f414c5d0 con 0x7fc6f40bebc0 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41dde10 con 0x7fc6f406ba30 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6fa7b1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f414dce0 con 0x7fc6f40be7a0 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.141+0000 7fc6f8d4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f40bebc0 0x7fc6f41e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f40bebc0 0x7fc6f41e3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50064/0 (socket says 172.21.15.49:50064) 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 -- 172.21.15.49:0/2871174242 learned_addr learned my addr 172.21.15.49:0/2871174242 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f37fe640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f40be7a0 0x7fc6f414c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f3fff640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6f406ba30 0x7fc6f40a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 -- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f40be7a0 msgr2=0x7fc6f414c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f40be7a0 0x7fc6f414c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 -- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6f406ba30 msgr2=0x7fc6f40a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6f406ba30 0x7fc6f40a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 -- 172.21.15.49:0/2871174242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4009400 con 0x7fc6f40bebc0 2024-03-14T22:05:24.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.142+0000 7fc6f8d4e640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f40bebc0 0x7fc6f41e3b70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c7d0 tx=0x7fc6ec00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.230+0000 7fc6f0ff9640 1 -- 172.21.15.49:0/2871174242 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec003b70 con 0x7fc6f40bebc0 2024-03-14T22:05:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.230+0000 7fc6f0ff9640 1 -- 172.21.15.49:0/2871174242 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec003cf0 con 0x7fc6f40bebc0 2024-03-14T22:05:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.230+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f414dfc0 con 0x7fc6f40bebc0 2024-03-14T22:05:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.230+0000 7fc6f0ff9640 1 -- 172.21.15.49:0/2871174242 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00aa90 con 0x7fc6f40bebc0 2024-03-14T22:05:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.230+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f414e580 con 0x7fc6f40bebc0 2024-03-14T22:05:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.231+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f40bbc10 con 0x7fc6f40bebc0 2024-03-14T22:05:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.232+0000 7fc6f0ff9640 1 -- 172.21.15.49:0/2871174242 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc6ec00abf0 con 0x7fc6f40bebc0 2024-03-14T22:05:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.232+0000 7fc6f0ff9640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc6e006a7a0 0x7fc6e006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.232+0000 7fc6f0ff9640 1 -- 172.21.15.49:0/2871174242 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc6ec088c20 con 0x7fc6f40bebc0 2024-03-14T22:05:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.233+0000 7fc6f3fff640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc6e006a7a0 0x7fc6e006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.233+0000 7fc6f3fff640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc6e006a7a0 0x7fc6e006cc60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4036000 tx=0x7fc6e403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.236+0000 7fc6f0ff9640 1 -- 172.21.15.49:0/2871174242 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc6ec053fd0 con 0x7fc6f40bebc0 2024-03-14T22:05:24.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.415+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6f4066bb0 con 0x7fc6f40bebc0 2024-03-14T22:05:24.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.415+0000 7fc6f0ff9640 1 -- 172.21.15.49:0/2871174242 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6ec003e60 con 0x7fc6f40bebc0 2024-03-14T22:05:24.414 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-14T22:05:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc6e006a7a0 msgr2=0x7fc6e006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc6e006a7a0 0x7fc6e006cc60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4036000 tx=0x7fc6e403d040 comp rx=0 tx=0).stop 2024-03-14T22:05:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f40bebc0 msgr2=0x7fc6f41e3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f40bebc0 0x7fc6f41e3b70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c7d0 tx=0x7fc6ec00cc90 comp rx=0 tx=0).stop 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 shutdown_connections 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc6e006a7a0 0x7fc6e006cc60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6f40bebc0 0x7fc6f41e3b70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6f40be7a0 0x7fc6f414c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 --2- 172.21.15.49:0/2871174242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6f406ba30 0x7fc6f40a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 >> 172.21.15.49:0/2871174242 conn(0x7fc6f40c0290 msgr2=0x7fc6f406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 shutdown_connections 2024-03-14T22:05:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.417+0000 7fc6fa7b1640 1 -- 172.21.15.49:0/2871174242 wait complete. 2024-03-14T22:05:24.427 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T22:05:24.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.537+0000 7fce911c4640 1 Processor -- start 2024-03-14T22:05:24.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.537+0000 7fce911c4640 1 -- start start 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce911c4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce8c14b3c0 0x7fce8c1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce911c4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c14edc0 con 0x7fce8c14df70 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce911c4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c0b9350 con 0x7fce8c144e40 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce911c4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c145220 con 0x7fce8c14b3c0 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce8a59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fce8c14df70 0x7fce8c14acb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43546/0 (socket says 172.21.15.49:43546) 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce8a59c640 1 -- 172.21.15.49:0/3926540013 learned_addr learned my addr 172.21.15.49:0/3926540013 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce8b59e640 1 --2- 172.21.15.49:0/3926540013 >> v2:172.21.15.97:3300/0 conn(0x7fce8c14b3c0 0x7fce8c1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431769016 0 0) 0x7fce8c14edc0 con 0x7fce8c14df70 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.538+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce74003520 con 0x7fce8c14df70 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991820508 0 0) 0x7fce74003520 con 0x7fce8c14df70 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 >> v2:172.21.15.97:3300/0 conn(0x7fce8c14b3c0 msgr2=0x7fce8c1562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 --2- 172.21.15.49:0/3926540013 >> v2:172.21.15.97:3300/0 conn(0x7fce8c14b3c0 0x7fce8c1562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 >> v1:172.21.15.97:6789/0 conn(0x7fce8c144e40 legacy=0x7fce8c0b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c14e490 con 0x7fce8c14df70 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7fce80001670 con 0x7fce8c14df70 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 >> v1:172.21.15.49:6789/0 conn(0x7fce8c14df70 legacy=0x7fce8c14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce740047c0 0x7fce74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce74005180 0x7fce7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7400c000 con 0x7fce74003970 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7400c170 con 0x7fce740047c0 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7400c330 con 0x7fce74005180 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8ad9d640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8a59c640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce740047c0 0x7fce74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8b59e640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce74005180 0x7fce7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8ad9d640 1 -- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce74005180 msgr2=0x7fce7400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8ad9d640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce74005180 0x7fce7400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8ad9d640 1 -- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce740047c0 msgr2=0x7fce74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8ad9d640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce740047c0 0x7fce74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8b59e640 1 -- 172.21.15.49:0/3926540013 reap_dead start 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.539+0000 7fce8ad9d640 1 -- 172.21.15.49:0/3926540013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74003520 con 0x7fce74003970 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.540+0000 7fce8ad9d640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce74003d90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce84001750 tx=0x7fce740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e11b5c42155b22 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.540+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce840052d0 con 0x7fce74003970 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.540+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8402e050 con 0x7fce74003970 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.540+0000 7fce89d9b640 1 -- 172.21.15.49:0/3926540013 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84003830 con 0x7fce74003970 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.540+0000 7fce911c4640 1 -- 172.21.15.49:0/3926540013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 msgr2=0x7fce74003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.540+0000 7fce911c4640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce74003d90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fce84001750 tx=0x7fce740057f0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.541+0000 7fce911c4640 1 -- 172.21.15.49:0/3926540013 shutdown_connections 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.541+0000 7fce911c4640 1 --2- 172.21.15.49:0/3926540013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce74003d90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.541+0000 7fce911c4640 1 -- 172.21.15.49:0/3926540013 >> 172.21.15.49:0/3926540013 conn(0x7fce8c0b9d30 msgr2=0x7fce8c0ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.541+0000 7fce911c4640 1 -- 172.21.15.49:0/3926540013 shutdown_connections 2024-03-14T22:05:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.541+0000 7fce911c4640 1 -- 172.21.15.49:0/3926540013 wait complete. 2024-03-14T22:05:24.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.542+0000 7fce911c4640 1 Processor -- start 2024-03-14T22:05:24.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.542+0000 7fce911c4640 1 -- start start 2024-03-14T22:05:24.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce8c143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce8c0b3480 0x7fce8c142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c142880 0x7fce8c1e3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c0a1b20 con 0x7fce74003970 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c0a2380 con 0x7fce8c142880 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c142d00 con 0x7fce8c0b3480 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce8c143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce8c143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50078/0 (socket says 172.21.15.49:50078) 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 -- 172.21.15.49:0/741564123 learned_addr learned my addr 172.21.15.49:0/741564123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8a59c640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce8c0b3480 0x7fce8c142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8b59e640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c142880 0x7fce8c1e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 -- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce8c0b3480 msgr2=0x7fce8c142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce8c0b3480 0x7fce8c142340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 -- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c142880 msgr2=0x7fce8c1e3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c142880 0x7fce8c1e3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 -- 172.21.15.49:0/741564123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c09fb20 con 0x7fce74003970 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce8ad9d640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce8c143a90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce84034040 tx=0x7fce84001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce737fe640 1 -- 172.21.15.49:0/741564123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84007a30 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce737fe640 1 -- 172.21.15.49:0/741564123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce840069e0 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c144170 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce737fe640 1 -- 172.21.15.49:0/741564123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84006d70 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.543+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c0a2010 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.544+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c073540 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.545+0000 7fce737fe640 1 -- 172.21.15.49:0/741564123 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fce840035f0 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.545+0000 7fce737fe640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fce7806a7a0 0x7fce7806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.545+0000 7fce737fe640 1 -- 172.21.15.49:0/741564123 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fce840aad50 con 0x7fce74003970 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.545+0000 7fce8a59c640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fce7806a7a0 0x7fce7806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.546+0000 7fce8a59c640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fce7806a7a0 0x7fce7806cc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce8c06ea80 tx=0x7fce80000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.549+0000 7fce737fe640 1 -- 172.21.15.49:0/741564123 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fce840ae030 con 0x7fce74003970 2024-03-14T22:05:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.729+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce8c0a1b20 con 0x7fce74003970 2024-03-14T22:05:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.730+0000 7fce737fe640 1 -- 172.21.15.49:0/741564123 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce84076230 con 0x7fce74003970 2024-03-14T22:05:24.729 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fce7806a7a0 msgr2=0x7fce7806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fce7806a7a0 0x7fce7806cc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce8c06ea80 tx=0x7fce80000ed0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 msgr2=0x7fce8c143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce8c143a90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fce84034040 tx=0x7fce84001120 comp rx=0 tx=0).stop 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 shutdown_connections 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fce7806a7a0 0x7fce7806cc60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c142880 0x7fce8c1e3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fce8c0b3480 0x7fce8c142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 --2- 172.21.15.49:0/741564123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce74003970 0x7fce8c143a90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 >> 172.21.15.49:0/741564123 conn(0x7fce8c0b9d30 msgr2=0x7fce8c148870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:24.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 shutdown_connections 2024-03-14T22:05:24.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.732+0000 7fce911c4640 1 -- 172.21.15.49:0/741564123 wait complete. 2024-03-14T22:05:24.742 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T22:05:24.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.850+0000 7f3720d18640 1 Processor -- start 2024-03-14T22:05:24.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.850+0000 7f3720d18640 1 -- start start 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f3720d18640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f371c14b820 0x7f371c06a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f3720d18640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c0b3a20 con 0x7f371c0bb1c0 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f3720d18640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c14f090 con 0x7f371c0bad60 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f3720d18640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c0bb5f0 con 0x7f371c14b820 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371ad9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f371c0bb1c0 0x7f371c14e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43550/0 (socket says 172.21.15.49:43550) 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371ad9d640 1 -- 172.21.15.49:0/1448843835 learned_addr learned my addr 172.21.15.49:0/1448843835 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371a59c640 1 --2- 172.21.15.49:0/1448843835 >> v2:172.21.15.97:3300/0 conn(0x7f371c14b820 0x7f371c06a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74385241 0 0) 0x7f371c0b3a20 con 0x7f371c0bb1c0 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3708003520 con 0x7f371c0bb1c0 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3274500448 0 0) 0x7f3708003520 con 0x7f371c0bb1c0 2024-03-14T22:05:24.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 >> v2:172.21.15.97:3300/0 conn(0x7f371c14b820 msgr2=0x7f371c06a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 --2- 172.21.15.49:0/1448843835 >> v2:172.21.15.97:3300/0 conn(0x7f371c14b820 0x7f371c06a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 >> v1:172.21.15.97:6789/0 conn(0x7f371c0bad60 legacy=0x7f371c14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0ba550 con 0x7f371c0bb1c0 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.851+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f370c0019f0 con 0x7f371c0bb1c0 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 >> v1:172.21.15.49:6789/0 conn(0x7f371c0bb1c0 legacy=0x7f371c14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371959a640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3708003970 0x7f3708003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371959a640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37080047c0 0x7f3708004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371959a640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3708005180 0x7f370800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0bb5f0 con 0x7f37080047c0 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370800c020 con 0x7f3708005180 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370800c1e0 con 0x7f3708003970 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f3719d9b640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37080047c0 0x7f3708004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371a59c640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3708003970 0x7f3708003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371ad9d640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3708005180 0x7f370800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f3719d9b640 1 -- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3708003970 msgr2=0x7f3708003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f3719d9b640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3708003970 0x7f3708003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f3719d9b640 1 -- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3708005180 msgr2=0x7f370800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f3719d9b640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3708005180 0x7f370800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f3719d9b640 1 -- 172.21.15.49:0/1448843835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708003520 con 0x7f37080047c0 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.852+0000 7f371ad9d640 1 -- 172.21.15.49:0/1448843835 reap_dead start 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.853+0000 7f3719d9b640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37080047c0 0x7f3708004c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3704001750 tx=0x7f37080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938af11cbf8dec45 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.853+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040052d0 con 0x7f37080047c0 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.853+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370402e050 con 0x7f37080047c0 2024-03-14T22:05:24.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.853+0000 7f371959a640 1 -- 172.21.15.49:0/1448843835 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704003830 con 0x7f37080047c0 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.853+0000 7f3720d18640 1 -- 172.21.15.49:0/1448843835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37080047c0 msgr2=0x7f3708004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.853+0000 7f3720d18640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37080047c0 0x7f3708004c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3704001750 tx=0x7f37080057f0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.854+0000 7f3720d18640 1 -- 172.21.15.49:0/1448843835 shutdown_connections 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.854+0000 7f3720d18640 1 --2- 172.21.15.49:0/1448843835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37080047c0 0x7f3708004c40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.854+0000 7f3720d18640 1 -- 172.21.15.49:0/1448843835 >> 172.21.15.49:0/1448843835 conn(0x7f371c0bcd80 msgr2=0x7f371c0bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.854+0000 7f3720d18640 1 -- 172.21.15.49:0/1448843835 shutdown_connections 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.854+0000 7f3720d18640 1 -- 172.21.15.49:0/1448843835 wait complete. 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 Processor -- start 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 -- start start 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37080047c0 0x7f371c0a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f371c1e0aa0 0x7f371c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c0b40d0 0x7f371c146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0a2760 con 0x7f371c0b40d0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1dfdd0 con 0x7f37080047c0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f3720d18640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0a2950 con 0x7f371c1e0aa0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.855+0000 7f371ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c0b40d0 0x7f371c146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c0b40d0 0x7f371c146020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50090/0 (socket says 172.21.15.49:50090) 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 -- 172.21.15.49:0/2672010647 learned_addr learned my addr 172.21.15.49:0/2672010647 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371a59c640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37080047c0 0x7f371c0a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f3719d9b640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f371c1e0aa0 0x7f371c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 -- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f371c1e0aa0 msgr2=0x7f371c0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f371c1e0aa0 0x7f371c0b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 -- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37080047c0 msgr2=0x7f371c0a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37080047c0 0x7f371c0a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 -- 172.21.15.49:0/2672010647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0b5470 con 0x7f371c0b40d0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f371ad9d640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c0b40d0 0x7f371c146020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f371c06cee0 tx=0x7f370c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f3712ffd640 1 -- 172.21.15.49:0/2672010647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c00aab0 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f3712ffd640 1 -- 172.21.15.49:0/2672010647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c00ac10 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c0b4a30 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f3712ffd640 1 -- 172.21.15.49:0/2672010647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c0133f0 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.856+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c1dffc0 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.857+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c0674d0 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.857+0000 7f3712ffd640 1 -- 172.21.15.49:0/2672010647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f370c013550 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.858+0000 7f3712ffd640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f36fc06a7a0 0x7f36fc06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.858+0000 7f3712ffd640 1 -- 172.21.15.49:0/2672010647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f370c0893f0 con 0x7f371c0b40d0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.858+0000 7f371a59c640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f36fc06a7a0 0x7f36fc06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:24.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.858+0000 7f371a59c640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f36fc06a7a0 0x7f36fc06cc60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3714009a00 tx=0x7f3714008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:24.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:24.862+0000 7f3712ffd640 1 -- 172.21.15.49:0/2672010647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f370c054820 con 0x7f371c0b40d0 2024-03-14T22:05:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.054+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f371c066bb0 con 0x7f371c0b40d0 2024-03-14T22:05:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.054+0000 7f3712ffd640 1 -- 172.21.15.49:0/2672010647 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f370c057e40 con 0x7f371c0b40d0 2024-03-14T22:05:25.053 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-03-14T22:05:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f36fc06a7a0 msgr2=0x7f36fc06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f36fc06a7a0 0x7f36fc06cc60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3714009a00 tx=0x7f3714008040 comp rx=0 tx=0).stop 2024-03-14T22:05:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c0b40d0 msgr2=0x7f371c146020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c0b40d0 0x7f371c146020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f371c06cee0 tx=0x7f370c007b00 comp rx=0 tx=0).stop 2024-03-14T22:05:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 shutdown_connections 2024-03-14T22:05:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f36fc06a7a0 0x7f36fc06cc60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:25.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f371c0b40d0 0x7f371c146020 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f371c1e0aa0 0x7f371c0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 --2- 172.21.15.49:0/2672010647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37080047c0 0x7f371c0a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 >> 172.21.15.49:0/2672010647 conn(0x7f371c0bcd80 msgr2=0x7f371c0b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 shutdown_connections 2024-03-14T22:05:25.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:25.056+0000 7f3720d18640 1 -- 172.21.15.49:0/2672010647 wait complete. 2024-03-14T22:05:25.066 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T22:05:25.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T22:05:25.066 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T22:05:26.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-03-14T22:05:26.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T22:05:26.003 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T22:05:26.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T22:05:26.045 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-03-14T22:05:26.134 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T22:05:26.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T22:05:26.135 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T22:05:26.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T22:05:26.227 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-03-14T22:05:26.357 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-14T22:05:26.368 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-14T22:05:26.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T22:05:26.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.479+0000 7f71a46a0640 1 Processor -- start 2024-03-14T22:05:26.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.479+0000 7f71a46a0640 1 -- start start 2024-03-14T22:05:26.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a46a0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c142cf0 0x7f719c0ba600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a46a0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f719c0b3260 0x7f719c06dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a46a0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f719c06e6a0 0x7f719c14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a46a0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719c0bab40 con 0x7f719c06e6a0 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a46a0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719c0b3740 con 0x7f719c0b3260 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a46a0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0b4ec0 con 0x7f719c142cf0 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a2c3d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f719c06e6a0 0x7f719c14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a2c3d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f719c06e6a0 0x7f719c14bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50096/0 (socket says 172.21.15.49:50096) 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.480+0000 7f71a2c3d640 1 -- 172.21.15.49:0/3228453978 learned_addr learned my addr 172.21.15.49:0/3228453978 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a1c3b640 1 --2- 172.21.15.49:0/3228453978 >> v2:172.21.15.97:3300/0 conn(0x7f719c0b3260 0x7f719c06dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a2c3d640 1 -- 172.21.15.49:0/3228453978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c142cf0 msgr2=0x7f719c0ba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a243c640 1 --2- 172.21.15.49:0/3228453978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c142cf0 0x7f719c0ba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a2c3d640 1 --2- 172.21.15.49:0/3228453978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c142cf0 0x7f719c0ba600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a2c3d640 1 -- 172.21.15.49:0/3228453978 >> v2:172.21.15.97:3300/0 conn(0x7f719c0b3260 msgr2=0x7f719c06dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a2c3d640 1 --2- 172.21.15.49:0/3228453978 >> v2:172.21.15.97:3300/0 conn(0x7f719c0b3260 0x7f719c06dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a2c3d640 1 -- 172.21.15.49:0/3228453978 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0b51a0 con 0x7f719c06e6a0 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.481+0000 7f71a1c3b640 1 --2- 172.21.15.49:0/3228453978 >> v2:172.21.15.97:3300/0 conn(0x7f719c0b3260 0x7f719c06dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a2c3d640 1 --2- 172.21.15.49:0/3228453978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c06e6a0 0x7f719c14bd90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f719c072180 tx=0x7f719402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1578b414f942ca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a143a640 1 -- 172.21.15.49:0/3228453978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719402ca70 con 0x7f719c06e6a0 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a143a640 1 -- 172.21.15.49:0/3228453978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719402cbd0 con 0x7f719c06e6a0 2024-03-14T22:05:26.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a143a640 1 -- 172.21.15.49:0/3228453978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940343f0 con 0x7f719c06e6a0 2024-03-14T22:05:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 -- 172.21.15.49:0/3228453978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c06e6a0 msgr2=0x7f719c14bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 --2- 172.21.15.49:0/3228453978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c06e6a0 0x7f719c14bd90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f719c072180 tx=0x7f719402bed0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 -- 172.21.15.49:0/3228453978 shutdown_connections 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 --2- 172.21.15.49:0/3228453978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c06e6a0 0x7f719c14bd90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 --2- 172.21.15.49:0/3228453978 >> v2:172.21.15.97:3300/0 conn(0x7f719c0b3260 0x7f719c06dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 --2- 172.21.15.49:0/3228453978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c142cf0 0x7f719c0ba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 -- 172.21.15.49:0/3228453978 >> 172.21.15.49:0/3228453978 conn(0x7f719c0bad40 msgr2=0x7f719c0bb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 -- 172.21.15.49:0/3228453978 shutdown_connections 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.482+0000 7f71a46a0640 1 -- 172.21.15.49:0/3228453978 wait complete. 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.483+0000 7f71a46a0640 1 Processor -- start 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a46a0640 1 -- start start 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a46a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f719c06e6a0 0x7f719c14a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a46a0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c0b3260 0x7f719c0a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a46a0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c142cf0 0x7f719c0a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a46a0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c14a6f0 con 0x7f719c142cf0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a46a0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0a24b0 con 0x7f719c06e6a0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a46a0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c14ba90 con 0x7f719c0b3260 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a2c3d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c142cf0 0x7f719c0a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a2c3d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c142cf0 0x7f719c0a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50104/0 (socket says 172.21.15.49:50104) 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a2c3d640 1 -- 172.21.15.49:0/4137917892 learned_addr learned my addr 172.21.15.49:0/4137917892 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a1c3b640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c0b3260 0x7f719c0a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.484+0000 7f71a243c640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f719c06e6a0 0x7f719c14a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a2c3d640 1 -- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c0b3260 msgr2=0x7f719c0a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a2c3d640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c0b3260 0x7f719c0a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a2c3d640 1 -- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f719c06e6a0 msgr2=0x7f719c14a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a2c3d640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f719c06e6a0 0x7f719c14a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a2c3d640 1 -- 172.21.15.49:0/4137917892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194009b30 con 0x7f719c142cf0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a2c3d640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c142cf0 0x7f719c0a2e80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7194036040 tx=0x7f7194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f718effd640 1 -- 172.21.15.49:0/4137917892 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719403e450 con 0x7f719c142cf0 2024-03-14T22:05:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f718effd640 1 -- 172.21.15.49:0/4137917892 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719403e5d0 con 0x7f719c142cf0 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f718effd640 1 -- 172.21.15.49:0/4137917892 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719403e980 con 0x7f719c142cf0 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c14bd70 con 0x7f719c142cf0 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.485+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c14b130 con 0x7f719c142cf0 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.486+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c0b43f0 con 0x7f719c142cf0 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.486+0000 7f718effd640 1 -- 172.21.15.49:0/4137917892 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f719403eae0 con 0x7f719c142cf0 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.487+0000 7f718effd640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f718006a7a0 0x7f718006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.487+0000 7f718effd640 1 -- 172.21.15.49:0/4137917892 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f71940abc70 con 0x7f719c142cf0 2024-03-14T22:05:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.487+0000 7f71a243c640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f718006a7a0 0x7f718006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.488+0000 7f71a243c640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f718006a7a0 0x7f718006cc60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f719000b7c0 tx=0x7f7190008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.491+0000 7f718effd640 1 -- 172.21.15.49:0/4137917892 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7194077010 con 0x7f719c142cf0 2024-03-14T22:05:26.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.667+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f719c06eb20 con 0x7f718006a7a0 2024-03-14T22:05:26.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.676+0000 7f718effd640 1 -- 172.21.15.49:0/4137917892 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292078 (secure 0 0 0) 0x7f719000c040 con 0x7f718006a7a0 2024-03-14T22:05:26.678 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:05:26.679 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T22:05:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f718006a7a0 msgr2=0x7f718006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f718006a7a0 0x7f718006cc60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f719000b7c0 tx=0x7f7190008040 comp rx=0 tx=0).stop 2024-03-14T22:05:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c142cf0 msgr2=0x7f719c0a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c142cf0 0x7f719c0a2e80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7194036040 tx=0x7f7194004a40 comp rx=0 tx=0).stop 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 shutdown_connections 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f718006a7a0 0x7f718006cc60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f719c142cf0 0x7f719c0a2e80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f719c0b3260 0x7f719c0a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 --2- 172.21.15.49:0/4137917892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f719c06e6a0 0x7f719c14a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 >> 172.21.15.49:0/4137917892 conn(0x7f719c0bad40 msgr2=0x7f719c141c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 shutdown_connections 2024-03-14T22:05:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.678+0000 7f71a46a0640 1 -- 172.21.15.49:0/4137917892 wait complete. 2024-03-14T22:05:26.688 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":1564,"stamp":"2024-03-14T22:05:25.588581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257405714,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":4401,"num_read_kb":112072,"num_write":843853,"num_write_kb":9407839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896061,"ondisk_log_size":896061,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8147260,"kb_used_data":504700,"kb_used_omap":13,"kb_used_meta":7642482,"kb_avail":746827460,"statfs":{"total":773094113280,"available":764751319040,"internally_reserved":0,"allocated":516812800,"data_stored":515502560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":7825901656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11540782,"num_objects":-2092,"num_object_clones":0,"num_object_copies":-4184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2092,"num_whiteouts":0,"num_read":14,"num_read_kb":226,"num_write":2321,"num_write_kb":8782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006146"},"pg_stats":[{"pgid":"3.3f","version":"23'10328","reported_seq":10370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.391677+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T22:05:17.391677+0000","last_peered":"2024-03-14T22:05:17.391677+0000","last_clean":"2024-03-14T22:05:17.391677+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T22:05:17.391677+0000","last_undegraded":"2024-03-14T22:05:17.391677+0000","last_fullsized":"2024-03-14T22:05:17.391677+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:57:09.325211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8785,"num_write_kb":106962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'9093","reported_seq":9142,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.446982+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T22:05:17.446982+0000","last_peered":"2024-03-14T22:05:17.446982+0000","last_clean":"2024-03-14T22:05:17.446982+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T22:05:17.446982+0000","last_undegraded":"2024-03-14T22:05:17.446982+0000","last_fullsized":"2024-03-14T22:05:17.446982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9093,"log_dups_size":0,"ondisk_log_size":9093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":7584,"num_write_kb":103945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'11562","reported_seq":11603,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.356641+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T22:05:23.356641+0000","last_peered":"2024-03-14T22:05:23.356641+0000","last_clean":"2024-03-14T22:05:23.356641+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T22:05:23.356641+0000","last_undegraded":"2024-03-14T22:05:23.356641+0000","last_fullsized":"2024-03-14T22:05:23.356641+0000","mapping_epoch":19,"log_start":"23'1500","ondisk_log_start":"23'1500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:08:55.492956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1274,"num_write":9914,"num_write_kb":127294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'10095","reported_seq":10141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.609893+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T22:05:16.609893+0000","last_peered":"2024-03-14T22:05:16.609893+0000","last_clean":"2024-03-14T22:05:16.609893+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T22:05:16.609893+0000","last_undegraded":"2024-03-14T22:05:16.609893+0000","last_fullsized":"2024-03-14T22:05:16.609893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8407,"num_write_kb":110879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'9604","reported_seq":9638,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.547040+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T22:05:16.547040+0000","last_peered":"2024-03-14T22:05:16.547040+0000","last_clean":"2024-03-14T22:05:16.547040+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T22:05:16.547040+0000","last_undegraded":"2024-03-14T22:05:16.547040+0000","last_fullsized":"2024-03-14T22:05:16.547040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9604,"log_dups_size":0,"ondisk_log_size":9604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8008,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'9639","reported_seq":9679,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.400588+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T22:05:17.400588+0000","last_peered":"2024-03-14T22:05:17.400588+0000","last_clean":"2024-03-14T22:05:17.400588+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T22:05:17.400588+0000","last_undegraded":"2024-03-14T22:05:17.400588+0000","last_fullsized":"2024-03-14T22:05:17.400588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":8054,"num_write_kb":114907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10032","reported_seq":10068,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431473+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T22:05:17.431473+0000","last_peered":"2024-03-14T22:05:17.431473+0000","last_clean":"2024-03-14T22:05:17.431473+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T22:05:17.431473+0000","last_undegraded":"2024-03-14T22:05:17.431473+0000","last_fullsized":"2024-03-14T22:05:17.431473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":8423,"num_write_kb":104789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'11190","reported_seq":11230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.614200+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T22:05:16.614200+0000","last_peered":"2024-03-14T22:05:16.614200+0000","last_clean":"2024-03-14T22:05:16.614200+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T22:05:16.614200+0000","last_undegraded":"2024-03-14T22:05:16.614200+0000","last_fullsized":"2024-03-14T22:05:16.614200+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":9605,"num_write_kb":125652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'9011","reported_seq":9048,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.436028+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T22:05:17.436028+0000","last_peered":"2024-03-14T22:05:17.436028+0000","last_clean":"2024-03-14T22:05:17.436028+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T22:05:17.436028+0000","last_undegraded":"2024-03-14T22:05:17.436028+0000","last_fullsized":"2024-03-14T22:05:17.436028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9011,"log_dups_size":0,"ondisk_log_size":9011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7381,"num_write_kb":104568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'9532","reported_seq":9568,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.411246+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T22:05:17.411246+0000","last_peered":"2024-03-14T22:05:17.411246+0000","last_clean":"2024-03-14T22:05:17.411246+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T22:05:17.411246+0000","last_undegraded":"2024-03-14T22:05:17.411246+0000","last_fullsized":"2024-03-14T22:05:17.411246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7843,"num_write_kb":113016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'9221","reported_seq":9262,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.399007+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T22:05:17.399007+0000","last_peered":"2024-03-14T22:05:17.399007+0000","last_clean":"2024-03-14T22:05:17.399007+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T22:05:17.399007+0000","last_undegraded":"2024-03-14T22:05:17.399007+0000","last_fullsized":"2024-03-14T22:05:17.399007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7677,"num_write_kb":104837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'10300","reported_seq":10335,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431722+0000","last_change":"2024-03-14T21:14:03.034043+0000","last_active":"2024-03-14T22:05:17.431722+0000","last_peered":"2024-03-14T22:05:17.431722+0000","last_clean":"2024-03-14T22:05:17.431722+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T22:05:17.431722+0000","last_undegraded":"2024-03-14T22:05:17.431722+0000","last_fullsized":"2024-03-14T22:05:17.431722+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:33.787920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8730,"num_write_kb":105734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'8860","reported_seq":8898,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.540388+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T22:05:16.540388+0000","last_peered":"2024-03-14T22:05:16.540388+0000","last_clean":"2024-03-14T22:05:16.540388+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T22:05:16.540388+0000","last_undegraded":"2024-03-14T22:05:16.540388+0000","last_fullsized":"2024-03-14T22:05:16.540388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8860,"log_dups_size":0,"ondisk_log_size":8860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":7264,"num_write_kb":95419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'10404","reported_seq":10443,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.392084+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T22:05:17.392084+0000","last_peered":"2024-03-14T22:05:17.392084+0000","last_clean":"2024-03-14T22:05:17.392084+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T22:05:17.392084+0000","last_undegraded":"2024-03-14T22:05:17.392084+0000","last_fullsized":"2024-03-14T22:05:17.392084+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8793,"num_write_kb":111405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'9697","reported_seq":9734,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.610064+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T22:05:16.610064+0000","last_peered":"2024-03-14T22:05:16.610064+0000","last_clean":"2024-03-14T22:05:16.610064+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T22:05:16.610064+0000","last_undegraded":"2024-03-14T22:05:16.610064+0000","last_fullsized":"2024-03-14T22:05:16.610064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9697,"log_dups_size":0,"ondisk_log_size":9697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8012,"num_write_kb":95820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'9656","reported_seq":9704,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.455534+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T22:05:17.455534+0000","last_peered":"2024-03-14T22:05:17.455534+0000","last_clean":"2024-03-14T22:05:17.455534+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T22:05:17.455534+0000","last_undegraded":"2024-03-14T22:05:17.455534+0000","last_fullsized":"2024-03-14T22:05:17.455534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9656,"log_dups_size":0,"ondisk_log_size":9656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3572,"num_write":8076,"num_write_kb":107857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8658","reported_seq":8695,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.457733+0000","last_change":"2024-03-14T21:14:44.859268+0000","last_active":"2024-03-14T22:05:17.457733+0000","last_peered":"2024-03-14T22:05:17.457733+0000","last_clean":"2024-03-14T22:05:17.457733+0000","last_became_active":"2024-03-14T21:14:44.858906+0000","last_became_peered":"2024-03-14T21:14:44.858906+0000","last_unstale":"2024-03-14T22:05:17.457733+0000","last_undegraded":"2024-03-14T22:05:17.457733+0000","last_fullsized":"2024-03-14T22:05:17.457733+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8658,"log_dups_size":0,"ondisk_log_size":8658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:56:07.747779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1528,"num_write":7020,"num_write_kb":102847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10194","reported_seq":10230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.574269+0000","last_change":"2024-03-14T21:14:03.029823+0000","last_active":"2024-03-14T22:05:16.574269+0000","last_peered":"2024-03-14T22:05:16.574269+0000","last_clean":"2024-03-14T22:05:16.574269+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T22:05:16.574269+0000","last_undegraded":"2024-03-14T22:05:16.574269+0000","last_fullsized":"2024-03-14T22:05:16.574269+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:47.131679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8552,"num_write_kb":105487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'10515","reported_seq":10549,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.365482+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T22:05:23.365482+0000","last_peered":"2024-03-14T22:05:23.365482+0000","last_clean":"2024-03-14T22:05:23.365482+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T22:05:23.365482+0000","last_undegraded":"2024-03-14T22:05:23.365482+0000","last_fullsized":"2024-03-14T22:05:23.365482+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":118219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'10892","reported_seq":10930,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.430759+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T22:05:17.430759+0000","last_peered":"2024-03-14T22:05:17.430759+0000","last_clean":"2024-03-14T22:05:17.430759+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T22:05:17.430759+0000","last_undegraded":"2024-03-14T22:05:17.430759+0000","last_fullsized":"2024-03-14T22:05:17.430759+0000","mapping_epoch":19,"log_start":"23'800","ondisk_log_start":"23'800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":9315,"num_write_kb":117297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'10951","reported_seq":10997,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.442792+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T22:05:17.442792+0000","last_peered":"2024-03-14T22:05:17.442792+0000","last_clean":"2024-03-14T22:05:17.442792+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T22:05:17.442792+0000","last_undegraded":"2024-03-14T22:05:17.442792+0000","last_fullsized":"2024-03-14T22:05:17.442792+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3050,"num_write":9338,"num_write_kb":109552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10716","reported_seq":10752,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.349639+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T22:05:23.349639+0000","last_peered":"2024-03-14T22:05:23.349639+0000","last_clean":"2024-03-14T22:05:23.349639+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T22:05:23.349639+0000","last_undegraded":"2024-03-14T22:05:23.349639+0000","last_fullsized":"2024-03-14T22:05:23.349639+0000","mapping_epoch":19,"log_start":"23'700","ondisk_log_start":"23'700","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":9066,"num_write_kb":118743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'10557","reported_seq":10593,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.383547+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T22:05:17.383547+0000","last_peered":"2024-03-14T22:05:17.383547+0000","last_clean":"2024-03-14T22:05:17.383547+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T22:05:17.383547+0000","last_undegraded":"2024-03-14T22:05:17.383547+0000","last_fullsized":"2024-03-14T22:05:17.383547+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8950,"num_write_kb":109690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'11172","reported_seq":11213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.420018+0000","last_change":"2024-03-14T21:14:03.038694+0000","last_active":"2024-03-14T22:05:17.420018+0000","last_peered":"2024-03-14T22:05:17.420018+0000","last_clean":"2024-03-14T22:05:17.420018+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T22:05:17.420018+0000","last_undegraded":"2024-03-14T22:05:17.420018+0000","last_fullsized":"2024-03-14T22:05:17.420018+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1530,"num_write":9571,"num_write_kb":127549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'10188","reported_seq":10225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.442741+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T22:05:17.442741+0000","last_peered":"2024-03-14T22:05:17.442741+0000","last_clean":"2024-03-14T22:05:17.442741+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T22:05:17.442741+0000","last_undegraded":"2024-03-14T22:05:17.442741+0000","last_fullsized":"2024-03-14T22:05:17.442741+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":8582,"num_write_kb":110246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'12329","reported_seq":12370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.390032+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T22:05:17.390032+0000","last_peered":"2024-03-14T22:05:17.390032+0000","last_clean":"2024-03-14T22:05:17.390032+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T22:05:17.390032+0000","last_undegraded":"2024-03-14T22:05:17.390032+0000","last_fullsized":"2024-03-14T22:05:17.390032+0000","mapping_epoch":19,"log_start":"23'2300","ondisk_log_start":"23'2300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10697,"num_write_kb":121303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'9238","reported_seq":9281,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.393718+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T22:05:17.393718+0000","last_peered":"2024-03-14T22:05:17.393718+0000","last_clean":"2024-03-14T22:05:17.393718+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T22:05:17.393718+0000","last_undegraded":"2024-03-14T22:05:17.393718+0000","last_fullsized":"2024-03-14T22:05:17.393718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9238,"log_dups_size":0,"ondisk_log_size":9238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2628,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7599,"num_write_kb":100407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'10304","reported_seq":10339,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.430400+0000","last_change":"2024-03-14T21:14:03.040690+0000","last_active":"2024-03-14T22:05:17.430400+0000","last_peered":"2024-03-14T22:05:17.430400+0000","last_clean":"2024-03-14T22:05:17.430400+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T22:05:17.430400+0000","last_undegraded":"2024-03-14T22:05:17.430400+0000","last_fullsized":"2024-03-14T22:05:17.430400+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":108137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'11950","reported_seq":11998,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.459036+0000","last_change":"2024-03-14T21:14:44.746590+0000","last_active":"2024-03-14T22:05:17.459036+0000","last_peered":"2024-03-14T22:05:17.459036+0000","last_clean":"2024-03-14T22:05:17.459036+0000","last_became_active":"2024-03-14T21:14:44.746271+0000","last_became_peered":"2024-03-14T21:14:44.746271+0000","last_unstale":"2024-03-14T22:05:17.459036+0000","last_undegraded":"2024-03-14T22:05:17.459036+0000","last_fullsized":"2024-03-14T22:05:17.459036+0000","mapping_epoch":22,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:01:46.971606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":10306,"num_write_kb":120667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'11949","reported_seq":11991,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431315+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T22:05:17.431315+0000","last_peered":"2024-03-14T22:05:17.431315+0000","last_clean":"2024-03-14T22:05:17.431315+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T22:05:17.431315+0000","last_undegraded":"2024-03-14T22:05:17.431315+0000","last_fullsized":"2024-03-14T22:05:17.431315+0000","mapping_epoch":19,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10416,"num_write_kb":126678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'11288","reported_seq":11335,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.451854+0000","last_change":"2024-03-14T21:14:03.029847+0000","last_active":"2024-03-14T22:05:17.451854+0000","last_peered":"2024-03-14T22:05:17.451854+0000","last_clean":"2024-03-14T22:05:17.451854+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T22:05:17.451854+0000","last_undegraded":"2024-03-14T22:05:17.451854+0000","last_fullsized":"2024-03-14T22:05:17.451854+0000","mapping_epoch":19,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2806,"num_write":9622,"num_write_kb":118224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'8631","reported_seq":8668,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.386380+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T22:05:17.386380+0000","last_peered":"2024-03-14T22:05:17.386380+0000","last_clean":"2024-03-14T22:05:17.386380+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T22:05:17.386380+0000","last_undegraded":"2024-03-14T22:05:17.386380+0000","last_fullsized":"2024-03-14T22:05:17.386380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":7051,"num_write_kb":105885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'9047","reported_seq":9083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431703+0000","last_change":"2024-03-14T21:14:03.037915+0000","last_active":"2024-03-14T22:05:17.431703+0000","last_peered":"2024-03-14T22:05:17.431703+0000","last_clean":"2024-03-14T22:05:17.431703+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T22:05:17.431703+0000","last_undegraded":"2024-03-14T22:05:17.431703+0000","last_fullsized":"2024-03-14T22:05:17.431703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":7364,"num_write_kb":104642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'10861","reported_seq":10894,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422940+0000","last_change":"2024-03-14T21:14:02.655762+0000","last_active":"2024-03-14T22:05:23.422940+0000","last_peered":"2024-03-14T22:05:23.422940+0000","last_clean":"2024-03-14T22:05:23.422940+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T22:05:23.422940+0000","last_undegraded":"2024-03-14T22:05:23.422940+0000","last_fullsized":"2024-03-14T22:05:23.422940+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10855,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'9532","reported_seq":9571,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.556727+0000","last_change":"2024-03-14T21:14:03.033091+0000","last_active":"2024-03-14T22:05:16.556727+0000","last_peered":"2024-03-14T22:05:16.556727+0000","last_clean":"2024-03-14T22:05:16.556727+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T22:05:16.556727+0000","last_undegraded":"2024-03-14T22:05:16.556727+0000","last_fullsized":"2024-03-14T22:05:16.556727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":7923,"num_write_kb":107722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10915","reported_seq":10948,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.394449+0000","last_change":"2024-03-14T21:14:02.662899+0000","last_active":"2024-03-14T22:05:23.394449+0000","last_peered":"2024-03-14T22:05:23.394449+0000","last_clean":"2024-03-14T22:05:23.394449+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T22:05:23.394449+0000","last_undegraded":"2024-03-14T22:05:23.394449+0000","last_fullsized":"2024-03-14T22:05:23.394449+0000","mapping_epoch":18,"log_start":"23'900","ondisk_log_start":"23'900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":10937,"num_write_kb":77181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'7036","reported_seq":7070,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422556+0000","last_change":"2024-03-14T21:14:02.660574+0000","last_active":"2024-03-14T22:05:23.422556+0000","last_peered":"2024-03-14T22:05:23.422556+0000","last_clean":"2024-03-14T22:05:23.422556+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T22:05:23.422556+0000","last_undegraded":"2024-03-14T22:05:23.422556+0000","last_fullsized":"2024-03-14T22:05:23.422556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":7036,"log_dups_size":0,"ondisk_log_size":7036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7046,"num_write_kb":58697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'11484","reported_seq":11518,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.365328+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T22:05:23.365328+0000","last_peered":"2024-03-14T22:05:23.365328+0000","last_clean":"2024-03-14T22:05:23.365328+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T22:05:23.365328+0000","last_undegraded":"2024-03-14T22:05:23.365328+0000","last_fullsized":"2024-03-14T22:05:23.365328+0000","mapping_epoch":19,"log_start":"23'1400","ondisk_log_start":"23'1400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":9890,"num_write_kb":106254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'11076","reported_seq":11110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.352182+0000","last_change":"2024-03-14T21:14:02.661303+0000","last_active":"2024-03-14T22:05:17.352182+0000","last_peered":"2024-03-14T22:05:17.352182+0000","last_clean":"2024-03-14T22:05:17.352182+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T22:05:17.352182+0000","last_undegraded":"2024-03-14T22:05:17.352182+0000","last_fullsized":"2024-03-14T22:05:17.352182+0000","mapping_epoch":18,"log_start":"23'1000","ondisk_log_start":"23'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11078,"num_write_kb":81064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'10471","reported_seq":10515,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.565713+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T22:05:16.565713+0000","last_peered":"2024-03-14T22:05:16.565713+0000","last_clean":"2024-03-14T22:05:16.565713+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T22:05:16.565713+0000","last_undegraded":"2024-03-14T22:05:16.565713+0000","last_fullsized":"2024-03-14T22:05:16.565713+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2800,"num_write":8909,"num_write_kb":114509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8286","reported_seq":8925,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.553161+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T22:05:15.553161+0000","last_peered":"2024-03-14T22:05:15.553161+0000","last_clean":"2024-03-14T22:05:15.553161+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T22:05:15.553161+0000","last_undegraded":"2024-03-14T22:05:15.553161+0000","last_fullsized":"2024-03-14T22:05:15.553161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8286,"log_dups_size":0,"ondisk_log_size":8286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":607,"num_read_kb":4494,"num_write":8292,"num_write_kb":66794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10197","reported_seq":10235,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.586675+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T22:05:16.586675+0000","last_peered":"2024-03-14T22:05:16.586675+0000","last_clean":"2024-03-14T22:05:16.586675+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T22:05:16.586675+0000","last_undegraded":"2024-03-14T22:05:16.586675+0000","last_fullsized":"2024-03-14T22:05:16.586675+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8609,"num_write_kb":104201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'10690","reported_seq":10724,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.418804+0000","last_change":"2024-03-14T21:14:02.660384+0000","last_active":"2024-03-14T22:05:23.418804+0000","last_peered":"2024-03-14T22:05:23.418804+0000","last_clean":"2024-03-14T22:05:23.418804+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T22:05:23.418804+0000","last_undegraded":"2024-03-14T22:05:23.418804+0000","last_fullsized":"2024-03-14T22:05:23.418804+0000","mapping_epoch":18,"log_start":"23'600","ondisk_log_start":"23'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":92935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'9094","reported_seq":9131,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.421535+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T22:05:17.421535+0000","last_peered":"2024-03-14T22:05:17.421535+0000","last_clean":"2024-03-14T22:05:17.421535+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T22:05:17.421535+0000","last_undegraded":"2024-03-14T22:05:17.421535+0000","last_fullsized":"2024-03-14T22:05:17.421535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9094,"log_dups_size":0,"ondisk_log_size":9094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:57:33.155885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7455,"num_write_kb":104328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9143","reported_seq":9177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.618955+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T22:05:16.618955+0000","last_peered":"2024-03-14T22:05:16.618955+0000","last_clean":"2024-03-14T22:05:16.618955+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T22:05:16.618955+0000","last_undegraded":"2024-03-14T22:05:16.618955+0000","last_fullsized":"2024-03-14T22:05:16.618955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9156,"num_write_kb":60300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11326","reported_seq":11369,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.356317+0000","last_change":"2024-03-14T21:14:03.032533+0000","last_active":"2024-03-14T22:05:23.356317+0000","last_peered":"2024-03-14T22:05:23.356317+0000","last_clean":"2024-03-14T22:05:23.356317+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T22:05:23.356317+0000","last_undegraded":"2024-03-14T22:05:23.356317+0000","last_fullsized":"2024-03-14T22:05:23.356317+0000","mapping_epoch":19,"log_start":"23'1300","ondisk_log_start":"23'1300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:41.581386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2542,"num_write":9699,"num_write_kb":113170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'13351","reported_seq":13939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.410668+0000","last_change":"2024-03-14T21:14:02.659028+0000","last_active":"2024-03-14T22:05:23.410668+0000","last_peered":"2024-03-14T22:05:23.410668+0000","last_clean":"2024-03-14T22:05:23.410668+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T22:05:23.410668+0000","last_undegraded":"2024-03-14T22:05:23.410668+0000","last_fullsized":"2024-03-14T22:05:23.410668+0000","mapping_epoch":18,"log_start":"23'3300","ondisk_log_start":"23'3300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":557,"num_read_kb":4455,"num_write":13349,"num_write_kb":107014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'9102","reported_seq":9148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.342305+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T22:05:17.342305+0000","last_peered":"2024-03-14T22:05:17.342305+0000","last_clean":"2024-03-14T22:05:17.342305+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T22:05:17.342305+0000","last_undegraded":"2024-03-14T22:05:17.342305+0000","last_fullsized":"2024-03-14T22:05:17.342305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9102,"log_dups_size":0,"ondisk_log_size":9102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3318,"num_write":7536,"num_write_kb":95248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'8009","reported_seq":8043,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.437600+0000","last_change":"2024-03-14T21:14:02.663139+0000","last_active":"2024-03-14T22:05:23.437600+0000","last_peered":"2024-03-14T22:05:23.437600+0000","last_clean":"2024-03-14T22:05:23.437600+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T22:05:23.437600+0000","last_undegraded":"2024-03-14T22:05:23.437600+0000","last_fullsized":"2024-03-14T22:05:23.437600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8009,"log_dups_size":0,"ondisk_log_size":8009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:25:31.095452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8008,"num_write_kb":62700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'9642","reported_seq":9681,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.460002+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T22:05:17.460002+0000","last_peered":"2024-03-14T22:05:17.460002+0000","last_clean":"2024-03-14T22:05:17.460002+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T22:05:17.460002+0000","last_undegraded":"2024-03-14T22:05:17.460002+0000","last_fullsized":"2024-03-14T22:05:17.460002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9642,"log_dups_size":0,"ondisk_log_size":9642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8033,"num_write_kb":100651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5756","reported_seq":5791,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.375166+0000","last_change":"2024-03-14T21:14:02.659773+0000","last_active":"2024-03-14T22:05:23.375166+0000","last_peered":"2024-03-14T22:05:23.375166+0000","last_clean":"2024-03-14T22:05:23.375166+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T22:05:23.375166+0000","last_undegraded":"2024-03-14T22:05:23.375166+0000","last_fullsized":"2024-03-14T22:05:23.375166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5756,"log_dups_size":0,"ondisk_log_size":5756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5754,"num_write_kb":48600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'9761","reported_seq":9797,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.410634+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T22:05:17.410634+0000","last_peered":"2024-03-14T22:05:17.410634+0000","last_clean":"2024-03-14T22:05:17.410634+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T22:05:17.410634+0000","last_undegraded":"2024-03-14T22:05:17.410634+0000","last_fullsized":"2024-03-14T22:05:17.410634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8135,"num_write_kb":106503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5851","reported_seq":5886,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.412820+0000","last_change":"2024-03-14T21:14:02.663585+0000","last_active":"2024-03-14T22:05:23.412820+0000","last_peered":"2024-03-14T22:05:23.412820+0000","last_clean":"2024-03-14T22:05:23.412820+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T22:05:23.412820+0000","last_undegraded":"2024-03-14T22:05:23.412820+0000","last_fullsized":"2024-03-14T22:05:23.412820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5851,"log_dups_size":0,"ondisk_log_size":5851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":52920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'9078","reported_seq":9122,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.334095+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T22:05:17.334095+0000","last_peered":"2024-03-14T22:05:17.334095+0000","last_clean":"2024-03-14T22:05:17.334095+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T22:05:17.334095+0000","last_undegraded":"2024-03-14T22:05:17.334095+0000","last_fullsized":"2024-03-14T22:05:17.334095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9078,"log_dups_size":0,"ondisk_log_size":9078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2548,"num_write":7495,"num_write_kb":101746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8321","reported_seq":8356,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.432570+0000","last_change":"2024-03-14T21:14:02.654931+0000","last_active":"2024-03-14T22:05:23.432570+0000","last_peered":"2024-03-14T22:05:23.432570+0000","last_clean":"2024-03-14T22:05:23.432570+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T22:05:23.432570+0000","last_undegraded":"2024-03-14T22:05:23.432570+0000","last_fullsized":"2024-03-14T22:05:23.432570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629345,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":8334,"num_write_kb":67488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'10125","reported_seq":10160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.361153+0000","last_change":"2024-03-14T21:14:44.859568+0000","last_active":"2024-03-14T22:05:17.361153+0000","last_peered":"2024-03-14T22:05:17.361153+0000","last_clean":"2024-03-14T22:05:17.361153+0000","last_became_active":"2024-03-14T21:14:44.859185+0000","last_became_peered":"2024-03-14T21:14:44.859185+0000","last_unstale":"2024-03-14T22:05:17.361153+0000","last_undegraded":"2024-03-14T22:05:17.361153+0000","last_fullsized":"2024-03-14T22:05:17.361153+0000","mapping_epoch":22,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:20:25.220662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":8491,"num_write_kb":113136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"23'8046","reported_seq":8081,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430049+0000","last_change":"2024-03-14T21:14:02.663747+0000","last_active":"2024-03-14T22:05:23.430049+0000","last_peered":"2024-03-14T22:05:23.430049+0000","last_clean":"2024-03-14T22:05:23.430049+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T22:05:23.430049+0000","last_undegraded":"2024-03-14T22:05:23.430049+0000","last_fullsized":"2024-03-14T22:05:23.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:59:11.091842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8051,"num_write_kb":62569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'9547","reported_seq":9589,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.579367+0000","last_change":"2024-03-14T21:14:03.030867+0000","last_active":"2024-03-14T22:05:16.579367+0000","last_peered":"2024-03-14T22:05:16.579367+0000","last_clean":"2024-03-14T22:05:16.579367+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T22:05:16.579367+0000","last_undegraded":"2024-03-14T22:05:16.579367+0000","last_fullsized":"2024-03-14T22:05:16.579367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2042,"num_write":7926,"num_write_kb":109666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'10889","reported_seq":10924,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.438118+0000","last_change":"2024-03-14T21:14:02.659095+0000","last_active":"2024-03-14T22:05:23.438118+0000","last_peered":"2024-03-14T22:05:23.438118+0000","last_clean":"2024-03-14T22:05:23.438118+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T22:05:23.438118+0000","last_undegraded":"2024-03-14T22:05:23.438118+0000","last_fullsized":"2024-03-14T22:05:23.438118+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10896,"num_write_kb":88926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9010","reported_seq":9049,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.599835+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T22:05:16.599835+0000","last_peered":"2024-03-14T22:05:16.599835+0000","last_clean":"2024-03-14T22:05:16.599835+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T22:05:16.599835+0000","last_undegraded":"2024-03-14T22:05:16.599835+0000","last_fullsized":"2024-03-14T22:05:16.599835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9010,"log_dups_size":0,"ondisk_log_size":9010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":7443,"num_write_kb":93234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'11492","reported_seq":11527,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:03:35.733472+0000","last_change":"2024-03-14T21:14:02.663441+0000","last_active":"2024-03-14T22:03:35.733472+0000","last_peered":"2024-03-14T22:03:35.733472+0000","last_clean":"2024-03-14T22:03:35.733472+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T22:03:35.733472+0000","last_undegraded":"2024-03-14T22:03:35.733472+0000","last_fullsized":"2024-03-14T22:03:35.733472+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11486,"num_write_kb":85212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'9510","reported_seq":9550,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.408549+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T22:05:17.408549+0000","last_peered":"2024-03-14T22:05:17.408549+0000","last_clean":"2024-03-14T22:05:17.408549+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T22:05:17.408549+0000","last_undegraded":"2024-03-14T22:05:17.408549+0000","last_fullsized":"2024-03-14T22:05:17.408549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9510,"log_dups_size":0,"ondisk_log_size":9510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":7940,"num_write_kb":115635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8904","reported_seq":8939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421547+0000","last_change":"2024-03-14T21:14:02.653919+0000","last_active":"2024-03-14T22:05:23.421547+0000","last_peered":"2024-03-14T22:05:23.421547+0000","last_clean":"2024-03-14T22:05:23.421547+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T22:05:23.421547+0000","last_undegraded":"2024-03-14T22:05:23.421547+0000","last_fullsized":"2024-03-14T22:05:23.421547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8904,"log_dups_size":0,"ondisk_log_size":8904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8892,"num_write_kb":67382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'9418","reported_seq":9456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431971+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T22:05:17.431971+0000","last_peered":"2024-03-14T22:05:17.431971+0000","last_clean":"2024-03-14T22:05:17.431971+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T22:05:17.431971+0000","last_undegraded":"2024-03-14T22:05:17.431971+0000","last_fullsized":"2024-03-14T22:05:17.431971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9418,"log_dups_size":0,"ondisk_log_size":9418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7820,"num_write_kb":110543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:02:10.082485+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T22:02:10.082485+0000","last_peered":"2024-03-14T22:02:10.082485+0000","last_clean":"2024-03-14T22:02:10.082485+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T22:02:10.082485+0000","last_undegraded":"2024-03-14T22:02:10.082485+0000","last_fullsized":"2024-03-14T22:02:10.082485+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'8577","reported_seq":8614,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430551+0000","last_change":"2024-03-14T21:14:02.652881+0000","last_active":"2024-03-14T22:05:23.430551+0000","last_peered":"2024-03-14T22:05:23.430551+0000","last_clean":"2024-03-14T22:05:23.430551+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T22:05:23.430551+0000","last_undegraded":"2024-03-14T22:05:23.430551+0000","last_fullsized":"2024-03-14T22:05:23.430551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8577,"log_dups_size":0,"ondisk_log_size":8577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":67724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'11125","reported_seq":11164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.609325+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T22:05:16.609325+0000","last_peered":"2024-03-14T22:05:16.609325+0000","last_clean":"2024-03-14T22:05:16.609325+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T22:05:16.609325+0000","last_undegraded":"2024-03-14T22:05:16.609325+0000","last_fullsized":"2024-03-14T22:05:16.609325+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":9507,"num_write_kb":117178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'8676","reported_seq":8714,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421927+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T22:05:23.421927+0000","last_peered":"2024-03-14T22:05:23.421927+0000","last_clean":"2024-03-14T22:05:23.421927+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T22:05:23.421927+0000","last_undegraded":"2024-03-14T22:05:23.421927+0000","last_fullsized":"2024-03-14T22:05:23.421927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8676,"log_dups_size":0,"ondisk_log_size":8676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:46.126855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8672,"num_write_kb":67194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10241","reported_seq":10279,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.608108+0000","last_change":"2024-03-14T21:14:03.033195+0000","last_active":"2024-03-14T22:05:16.608108+0000","last_peered":"2024-03-14T22:05:16.608108+0000","last_clean":"2024-03-14T22:05:16.608108+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T22:05:16.608108+0000","last_undegraded":"2024-03-14T22:05:16.608108+0000","last_fullsized":"2024-03-14T22:05:16.608108+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8656,"num_write_kb":115905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11303","reported_seq":12007,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.404838+0000","last_change":"2024-03-14T21:14:02.658709+0000","last_active":"2024-03-14T22:05:23.404838+0000","last_peered":"2024-03-14T22:05:23.404838+0000","last_clean":"2024-03-14T22:05:23.404838+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T22:05:23.404838+0000","last_undegraded":"2024-03-14T22:05:23.404838+0000","last_fullsized":"2024-03-14T22:05:23.404838+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":669,"num_read_kb":4516,"num_write":11296,"num_write_kb":89038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'9435","reported_seq":9475,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.585185+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T22:05:16.585185+0000","last_peered":"2024-03-14T22:05:16.585185+0000","last_clean":"2024-03-14T22:05:16.585185+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T22:05:16.585185+0000","last_undegraded":"2024-03-14T22:05:16.585185+0000","last_fullsized":"2024-03-14T22:05:16.585185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9435,"log_dups_size":0,"ondisk_log_size":9435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7837,"num_write_kb":123513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'9296","reported_seq":9333,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.408089+0000","last_change":"2024-03-14T21:14:02.657401+0000","last_active":"2024-03-14T22:05:23.408089+0000","last_peered":"2024-03-14T22:05:23.408089+0000","last_clean":"2024-03-14T22:05:23.408089+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T22:05:23.408089+0000","last_undegraded":"2024-03-14T22:05:23.408089+0000","last_fullsized":"2024-03-14T22:05:23.408089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9294,"num_write_kb":71830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'9543","reported_seq":9580,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.459567+0000","last_change":"2024-03-14T21:14:03.036152+0000","last_active":"2024-03-14T22:05:17.459567+0000","last_peered":"2024-03-14T22:05:17.459567+0000","last_clean":"2024-03-14T22:05:17.459567+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T22:05:17.459567+0000","last_undegraded":"2024-03-14T22:05:17.459567+0000","last_fullsized":"2024-03-14T22:05:17.459567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9543,"log_dups_size":0,"ondisk_log_size":9543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:14.794102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":7936,"num_write_kb":109238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'11129","reported_seq":11166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.404373+0000","last_change":"2024-03-14T21:14:02.653065+0000","last_active":"2024-03-14T22:05:17.404373+0000","last_peered":"2024-03-14T22:05:17.404373+0000","last_clean":"2024-03-14T22:05:17.404373+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T22:05:17.404373+0000","last_undegraded":"2024-03-14T22:05:17.404373+0000","last_fullsized":"2024-03-14T22:05:17.404373+0000","mapping_epoch":18,"log_start":"23'1100","ondisk_log_start":"23'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11129,"num_write_kb":85841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'12164","reported_seq":12200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.438275+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T22:05:17.438275+0000","last_peered":"2024-03-14T22:05:17.438275+0000","last_clean":"2024-03-14T22:05:17.438275+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T22:05:17.438275+0000","last_undegraded":"2024-03-14T22:05:17.438275+0000","last_fullsized":"2024-03-14T22:05:17.438275+0000","mapping_epoch":19,"log_start":"23'2100","ondisk_log_start":"23'2100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10560,"num_write_kb":124816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'9283","reported_seq":9320,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.431083+0000","last_change":"2024-03-14T21:14:02.659612+0000","last_active":"2024-03-14T22:05:23.431083+0000","last_peered":"2024-03-14T22:05:23.431083+0000","last_clean":"2024-03-14T22:05:23.431083+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T22:05:23.431083+0000","last_undegraded":"2024-03-14T22:05:23.431083+0000","last_fullsized":"2024-03-14T22:05:23.431083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9283,"log_dups_size":0,"ondisk_log_size":9283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9294,"num_write_kb":71728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9769","reported_seq":9810,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.417912+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T22:05:17.417912+0000","last_peered":"2024-03-14T22:05:17.417912+0000","last_clean":"2024-03-14T22:05:17.417912+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T22:05:17.417912+0000","last_undegraded":"2024-03-14T22:05:17.417912+0000","last_fullsized":"2024-03-14T22:05:17.417912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9769,"log_dups_size":0,"ondisk_log_size":9769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8091,"num_write_kb":107879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'8352","reported_seq":8981,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:11.741680+0000","last_change":"2024-03-14T21:14:02.662122+0000","last_active":"2024-03-14T22:05:11.741680+0000","last_peered":"2024-03-14T22:05:11.741680+0000","last_clean":"2024-03-14T22:05:11.741680+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T22:05:11.741680+0000","last_undegraded":"2024-03-14T22:05:11.741680+0000","last_fullsized":"2024-03-14T22:05:11.741680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8352,"log_dups_size":0,"ondisk_log_size":8352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":593,"num_read_kb":4491,"num_write":9527,"num_write_kb":73065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'10070","reported_seq":10114,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.350964+0000","last_change":"2024-03-14T21:14:03.037921+0000","last_active":"2024-03-14T22:05:23.350964+0000","last_peered":"2024-03-14T22:05:23.350964+0000","last_clean":"2024-03-14T22:05:23.350964+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T22:05:23.350964+0000","last_undegraded":"2024-03-14T22:05:23.350964+0000","last_fullsized":"2024-03-14T22:05:23.350964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8532,"num_write_kb":114847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5880","reported_seq":5916,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.420296+0000","last_change":"2024-03-14T21:14:02.665745+0000","last_active":"2024-03-14T22:05:23.420296+0000","last_peered":"2024-03-14T22:05:23.420296+0000","last_clean":"2024-03-14T22:05:23.420296+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T22:05:23.420296+0000","last_undegraded":"2024-03-14T22:05:23.420296+0000","last_fullsized":"2024-03-14T22:05:23.420296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5880,"log_dups_size":0,"ondisk_log_size":5880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5885,"num_write_kb":44672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'10911","reported_seq":10953,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.453069+0000","last_change":"2024-03-14T21:14:03.029269+0000","last_active":"2024-03-14T22:05:17.453069+0000","last_peered":"2024-03-14T22:05:17.453069+0000","last_clean":"2024-03-14T22:05:17.453069+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T22:05:17.453069+0000","last_undegraded":"2024-03-14T22:05:17.453069+0000","last_fullsized":"2024-03-14T22:05:17.453069+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":9274,"num_write_kb":112928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'4360","reported_seq":4396,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.551031+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T22:05:15.551031+0000","last_peered":"2024-03-14T22:05:15.551031+0000","last_clean":"2024-03-14T22:05:15.551031+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T22:05:15.551031+0000","last_undegraded":"2024-03-14T22:05:15.551031+0000","last_fullsized":"2024-03-14T22:05:15.551031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4362,"num_write_kb":43693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'9883","reported_seq":9931,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.440163+0000","last_change":"2024-03-14T21:14:03.039549+0000","last_active":"2024-03-14T22:05:16.440163+0000","last_peered":"2024-03-14T22:05:16.440163+0000","last_clean":"2024-03-14T22:05:16.440163+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T22:05:16.440163+0000","last_undegraded":"2024-03-14T22:05:16.440163+0000","last_fullsized":"2024-03-14T22:05:16.440163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9883,"log_dups_size":0,"ondisk_log_size":9883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:23.510516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":8266,"num_write_kb":116331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'6399","reported_seq":6435,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.600135+0000","last_change":"2024-03-14T21:14:02.660292+0000","last_active":"2024-03-14T22:05:16.600135+0000","last_peered":"2024-03-14T22:05:16.600135+0000","last_clean":"2024-03-14T22:05:16.600135+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T22:05:16.600135+0000","last_undegraded":"2024-03-14T22:05:16.600135+0000","last_fullsized":"2024-03-14T22:05:16.600135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6399,"log_dups_size":0,"ondisk_log_size":6399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6417,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'10919","reported_seq":10954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.435069+0000","last_change":"2024-03-14T21:14:03.040817+0000","last_active":"2024-03-14T22:05:17.435069+0000","last_peered":"2024-03-14T22:05:17.435069+0000","last_clean":"2024-03-14T22:05:17.435069+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T22:05:17.435069+0000","last_undegraded":"2024-03-14T22:05:17.435069+0000","last_fullsized":"2024-03-14T22:05:17.435069+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":120976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'9349","reported_seq":9385,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:01:50.712636+0000","last_change":"2024-03-14T21:14:02.661073+0000","last_active":"2024-03-14T22:01:50.712636+0000","last_peered":"2024-03-14T22:01:50.712636+0000","last_clean":"2024-03-14T22:01:50.712636+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T22:01:50.712636+0000","last_undegraded":"2024-03-14T22:01:50.712636+0000","last_fullsized":"2024-03-14T22:01:50.712636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9349,"log_dups_size":0,"ondisk_log_size":9349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9372,"num_write_kb":71713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'10191","reported_seq":10229,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.350982+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T22:05:17.350982+0000","last_peered":"2024-03-14T22:05:17.350982+0000","last_clean":"2024-03-14T22:05:17.350982+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T22:05:17.350982+0000","last_undegraded":"2024-03-14T22:05:17.350982+0000","last_fullsized":"2024-03-14T22:05:17.350982+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8519,"num_write_kb":106953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'11400","reported_seq":11835,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.412968+0000","last_change":"2024-03-14T21:14:02.663131+0000","last_active":"2024-03-14T22:05:23.412968+0000","last_peered":"2024-03-14T22:05:23.412968+0000","last_clean":"2024-03-14T22:05:23.412968+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T22:05:23.412968+0000","last_undegraded":"2024-03-14T22:05:23.412968+0000","last_fullsized":"2024-03-14T22:05:23.412968+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13689487,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":401,"num_read_kb":1362,"num_write":11406,"num_write_kb":86543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'13457","reported_seq":13498,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.396876+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T22:05:17.396876+0000","last_peered":"2024-03-14T22:05:17.396876+0000","last_clean":"2024-03-14T22:05:17.396876+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T22:05:17.396876+0000","last_undegraded":"2024-03-14T22:05:17.396876+0000","last_fullsized":"2024-03-14T22:05:17.396876+0000","mapping_epoch":19,"log_start":"23'3400","ondisk_log_start":"23'3400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":11847,"num_write_kb":149794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'10672","reported_seq":10705,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.469190+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T22:05:16.469190+0000","last_peered":"2024-03-14T22:05:16.469190+0000","last_clean":"2024-03-14T22:05:16.469190+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T22:05:16.469190+0000","last_undegraded":"2024-03-14T22:05:16.469190+0000","last_fullsized":"2024-03-14T22:05:16.469190+0000","mapping_epoch":19,"log_start":"23'600","ondisk_log_start":"23'600","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":8986,"num_write_kb":111494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6864","reported_seq":6897,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.409531+0000","last_change":"2024-03-14T21:14:02.660393+0000","last_active":"2024-03-14T22:05:23.409531+0000","last_peered":"2024-03-14T22:05:23.409531+0000","last_clean":"2024-03-14T22:05:23.409531+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T22:05:23.409531+0000","last_undegraded":"2024-03-14T22:05:23.409531+0000","last_fullsized":"2024-03-14T22:05:23.409531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6864,"log_dups_size":0,"ondisk_log_size":6864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6885,"num_write_kb":66562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'11241","reported_seq":11278,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.351892+0000","last_change":"2024-03-14T21:14:44.859446+0000","last_active":"2024-03-14T22:05:17.351892+0000","last_peered":"2024-03-14T22:05:17.351892+0000","last_clean":"2024-03-14T22:05:17.351892+0000","last_became_active":"2024-03-14T21:14:44.859128+0000","last_became_peered":"2024-03-14T21:14:44.859128+0000","last_unstale":"2024-03-14T22:05:17.351892+0000","last_undegraded":"2024-03-14T22:05:17.351892+0000","last_fullsized":"2024-03-14T22:05:17.351892+0000","mapping_epoch":22,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:32:11.377675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1780,"num_write":9572,"num_write_kb":116659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15422","reported_seq":16524,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.386679+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T22:05:23.386679+0000","last_peered":"2024-03-14T22:05:23.386679+0000","last_clean":"2024-03-14T22:05:23.386679+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T22:05:23.386679+0000","last_undegraded":"2024-03-14T22:05:23.386679+0000","last_fullsized":"2024-03-14T22:05:23.386679+0000","mapping_epoch":18,"log_start":"23'5400","ondisk_log_start":"23'5400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:48.203059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1069,"num_read_kb":4840,"num_write":15404,"num_write_kb":112501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'10425","reported_seq":10472,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.447186+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T22:05:17.447186+0000","last_peered":"2024-03-14T22:05:17.447186+0000","last_clean":"2024-03-14T22:05:17.447186+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T22:05:17.447186+0000","last_undegraded":"2024-03-14T22:05:17.447186+0000","last_fullsized":"2024-03-14T22:05:17.447186+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2796,"num_write":8857,"num_write_kb":109660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'8706","reported_seq":8744,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.407645+0000","last_change":"2024-03-14T21:14:02.654963+0000","last_active":"2024-03-14T22:05:23.407645+0000","last_peered":"2024-03-14T22:05:23.407645+0000","last_clean":"2024-03-14T22:05:23.407645+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T22:05:23.407645+0000","last_undegraded":"2024-03-14T22:05:23.407645+0000","last_fullsized":"2024-03-14T22:05:23.407645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8706,"log_dups_size":0,"ondisk_log_size":8706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8698,"num_write_kb":67327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'11830","reported_seq":11868,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.429804+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T22:05:17.429804+0000","last_peered":"2024-03-14T22:05:17.429804+0000","last_clean":"2024-03-14T22:05:17.429804+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T22:05:17.429804+0000","last_undegraded":"2024-03-14T22:05:17.429804+0000","last_fullsized":"2024-03-14T22:05:17.429804+0000","mapping_epoch":19,"log_start":"23'1800","ondisk_log_start":"23'1800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":10179,"num_write_kb":107836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6046","reported_seq":6083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.408036+0000","last_change":"2024-03-14T21:14:02.656125+0000","last_active":"2024-03-14T22:05:23.408036+0000","last_peered":"2024-03-14T22:05:23.408036+0000","last_clean":"2024-03-14T22:05:23.408036+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T22:05:23.408036+0000","last_undegraded":"2024-03-14T22:05:23.408036+0000","last_fullsized":"2024-03-14T22:05:23.408036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6048,"num_write_kb":48854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2628,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":345,"num_read_kb":87814,"num_write":552755,"num_write_kb":7123402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624176,"ondisk_log_size":624176,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":256812718,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":3940,"num_read_kb":24158,"num_write":290939,"num_write_kb":2282021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":513675264,"data_stored":513612702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271768,"ondisk_log_size":271768,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575471,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252664,"kb_used_data":112496,"kb_used_omap":1,"kb_used_meta":1140158,"kb_avail":93119176,"statfs":{"total":96636764160,"available":95354036224,"internally_reserved":0,"allocated":115195904,"data_stored":115031492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1167522059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45500000000000002,"15min":0.48699999999999999},"min":{"1min":0.26100000000000001,"5min":0.23000000000000001,"15min":0.19500000000000001},"max":{"1min":0.57499999999999996,"5min":2.782,"15min":4.7999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.63900000000000001,"5min":2.8519999999999999,"15min":3.9239999999999999},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":0.66000000000000003,"5min":1.8520000000000001,"15min":3.903},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51100000000000001,"15min":0.55900000000000005},"min":{"1min":0.30099999999999999,"5min":0.28599999999999998,"15min":0.26400000000000001},"max":{"1min":0.60599999999999998,"5min":6.1440000000000001,"15min":6.6689999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":0.63500000000000001,"5min":3.8210000000000002,"15min":4.8310000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.59199999999999997,"15min":0.54700000000000004},"min":{"1min":0.32000000000000001,"5min":0.29799999999999999,"15min":0.25900000000000001},"max":{"1min":0.59799999999999998,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67200000000000004,"5min":6.1820000000000004,"15min":6.6909999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.5,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.622,"5min":1.9339999999999999,"15min":4.851},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.68700000000000006,"5min":1.226,"15min":6.7220000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.44700000000000001},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.23300000000000001},"max":{"1min":0.69899999999999995,"5min":1.492,"15min":5.2679999999999998},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42999999999999999,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.255,"15min":0.24199999999999999},"max":{"1min":0.71799999999999997,"5min":1.3089999999999999,"15min":3.8740000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.755,"5min":1.321,"15min":6.6459999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.49399999999999999},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69599999999999995,"5min":1.2529999999999999,"15min":4.157},"last":0.442},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.55800000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":1.2509999999999999,"15min":6.6109999999999998},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575470,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101820,"kb_used_data":69876,"kb_used_omap":1,"kb_used_meta":1031934,"kb_avail":93270020,"statfs":{"total":96636764160,"available":95508500480,"internally_reserved":0,"allocated":71553024,"data_stored":71391599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1056700683},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.47299999999999998,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.188},"max":{"1min":2.899,"5min":2.899,"15min":17.885999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.23000000000000001},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.52000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.216},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":7.226},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":1.224,"15min":7.1699999999999999},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.6189999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.26800000000000002},"max":{"1min":3.282,"5min":3.282,"15min":7.2489999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":1.2470000000000001,"15min":7.1470000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.61799999999999999,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.29899999999999999},"max":{"1min":6.1660000000000004,"5min":6.1660000000000004,"15min":6.1660000000000004},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.468,"5min":2.468,"15min":7.3330000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.49199999999999999,"15min":0.45300000000000001},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":6.0229999999999997,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":8.234,"5min":8.234,"15min":8.234},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48799999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.90600000000000003,"5min":1.3340000000000001,"15min":7.5380000000000003},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.54100000000000004,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":7.2039999999999997},"last":0.80800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575470,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1147640,"kb_used_data":69872,"kb_used_omap":1,"kb_used_meta":1077758,"kb_avail":93224200,"statfs":{"total":96636764160,"available":95461580800,"internally_reserved":0,"allocated":71548928,"data_stored":71391017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1103624459},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.55500000000000005},"min":{"1min":0.27300000000000002,"5min":0.221,"15min":0.192},"max":{"1min":0.71199999999999997,"5min":3.2149999999999999,"15min":7.0990000000000002},"last":0.438},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.57499999999999996},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.21099999999999999},"max":{"1min":0.66800000000000004,"5min":5.7359999999999998,"15min":12.281000000000001},"last":0.318}]},{"osd":1,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.68100000000000005,"15min":0.60399999999999998},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.20599999999999999},"max":{"1min":0.70299999999999996,"5min":14.952,"15min":14.952},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49299999999999999,"15min":0.54500000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.222},"max":{"1min":0.63600000000000001,"5min":3.1339999999999999,"15min":8.8539999999999992},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.75800000000000001,"15min":0.63100000000000001},"min":{"1min":0.28699999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":0.70599999999999996,"5min":18.623000000000001,"15min":18.623000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.96399999999999997,"15min":0.70699999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.67000000000000004,"5min":30.004999999999999,"15min":30.004999999999999},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.25},"max":{"1min":0.66200000000000003,"5min":3.6600000000000001,"15min":9.2309999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.57699999999999996},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":0.82699999999999996,"15min":7.3780000000000001},"last":0.498}]},{"osd":4,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.59899999999999998,"15min":0.53700000000000003},"min":{"1min":0.22700000000000001,"5min":0.219,"15min":0.20999999999999999},"max":{"1min":0.58399999999999996,"5min":12.414,"15min":12.414},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.496,"15min":0.57099999999999995},"min":{"1min":0.308,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60799999999999998,"5min":3.1970000000000001,"15min":8.9220000000000006},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.44600000000000001,"15min":0.46600000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.22},"max":{"1min":0.55600000000000005,"5min":1.905,"15min":7.4000000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.61599999999999999,"15min":0.58299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.24299999999999999},"max":{"1min":0.65700000000000003,"5min":10.278,"15min":10.278},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.52300000000000002,"15min":0.55100000000000005},"min":{"1min":0.27500000000000002,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":0.64200000000000002,"5min":3.9820000000000002,"15min":7.1619999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.28399999999999997},"max":{"1min":0.623,"5min":3.7789999999999999,"15min":7.3310000000000004},"last":0.57199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":961852,"kb_used_data":42292,"kb_used_omap":1,"kb_used_meta":919550,"kb_avail":93409988,"statfs":{"total":96636764160,"available":95651827712,"internally_reserved":0,"allocated":43307008,"data_stored":43137500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":941619467},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.495,"15min":0.55600000000000005},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":0.73799999999999999,"5min":2.23,"15min":15.013},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.26000000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.72599999999999998,"5min":2.1139999999999999,"15min":5.6879999999999997},"last":0.628}]},{"osd":1,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":1.806,"15min":4.6479999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.52000000000000002},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":0.64900000000000002,"5min":0.77800000000000002,"15min":4.343},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.503},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70699999999999996,"5min":0.97099999999999997,"15min":5.5540000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57499999999999996,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.26000000000000001},"max":{"1min":0.75700000000000001,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.5}]},{"osd":3,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.22},"max":{"1min":0.67700000000000005,"5min":2.1320000000000001,"15min":9.9800000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.33000000000000002,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":0.77200000000000002,"5min":1.8919999999999999,"15min":5.899},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41499999999999998,"15min":0.40999999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.20599999999999999},"max":{"1min":0.61099999999999999,"5min":0.71599999999999997,"15min":3.0470000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.41899999999999998,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.61699999999999999,"5min":0.69999999999999996,"15min":4.2460000000000004},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44500000000000001,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.76100000000000001,"5min":0.90900000000000003,"15min":3.8650000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45700000000000002,"15min":0.45900000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":1.5680000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.23400000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":8.7639999999999993},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.27400000000000002},"max":{"1min":0.69399999999999995,"5min":0.94799999999999995,"15min":3.7850000000000001},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1131324,"kb_used_data":57588,"kb_used_omap":1,"kb_used_meta":1073726,"kb_avail":93240516,"statfs":{"total":96636764160,"available":95478288384,"internally_reserved":0,"allocated":58970112,"data_stored":58808687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1099495691},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.57599999999999996,"15min":0.53000000000000003},"min":{"1min":0.23000000000000001,"5min":0.20499999999999999,"15min":0.17000000000000001},"max":{"1min":0.52300000000000002,"5min":7.7270000000000003,"15min":14.695},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.186},"max":{"1min":0.51900000000000002,"5min":7.9770000000000003,"15min":8.9649999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.378,"15min":0.45500000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":0.71799999999999997,"5min":0.93500000000000005,"15min":5.5289999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.53700000000000003,"15min":0.60699999999999998},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.20100000000000001},"max":{"1min":0.83899999999999997,"5min":6.1740000000000004,"15min":14.898},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.61599999999999999,"15min":0.57099999999999995},"min":{"1min":0.251,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.83099999999999996,"5min":9.4079999999999995,"15min":9.5150000000000006},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.55900000000000005},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":2.9430000000000001,"15min":15.263},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.71899999999999997,"15min":0.74399999999999999},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":0.77700000000000002,"5min":9.6080000000000005,"15min":14.071999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.622,"15min":0.69499999999999995},"min":{"1min":0.40200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":3.5710000000000002,"15min":14.109},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.70899999999999996,"15min":0.68200000000000005},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":11.443,"15min":11.443},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.74199999999999999,"15min":0.77000000000000002},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.78000000000000003,"5min":7.6399999999999997,"15min":14.550000000000001},"last":0.85399999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.82999999999999996,"15min":0.76500000000000001},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.29299999999999998},"max":{"1min":0.78400000000000003,"5min":9.7200000000000006,"15min":9.7200000000000006},"last":0.873},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.78200000000000003,"15min":0.69899999999999995},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":15.339,"15min":15.339},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.81599999999999995,"15min":0.77700000000000002},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":7.7030000000000003,"15min":14.726000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.83399999999999996,"15min":0.76300000000000001},"min":{"1min":0.433,"5min":0.379,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":9.5079999999999991,"15min":9.5079999999999991},"last":0.91400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575471,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649700,"kb_used_data":45148,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":93722140,"statfs":{"total":96636764160,"available":95971471360,"internally_reserved":0,"allocated":46231552,"data_stored":46077232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":619051275},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.61299999999999999,"15min":0.45800000000000002},"min":{"1min":0.24399999999999999,"5min":0.184,"15min":0.182},"max":{"1min":2.347,"5min":10.869,"15min":10.869},"last":0.311},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.67800000000000005,"15min":0.55200000000000005},"min":{"1min":0.23799999999999999,"5min":0.20100000000000001,"15min":0.189},"max":{"1min":1.107,"5min":9.6649999999999991,"15min":12.317},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.53100000000000003,"15min":0.45600000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":2.0049999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.20100000000000001},"max":{"1min":0.96799999999999997,"5min":7.8090000000000002,"15min":9.5429999999999993},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.66700000000000004,"15min":0.61899999999999999},"min":{"1min":0.26500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56499999999999995,"5min":13.130000000000001,"15min":27.355},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52800000000000002,"15min":0.57099999999999995},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.26000000000000001},"max":{"1min":1.929,"5min":3.4390000000000001,"15min":12.451000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83099999999999996,"15min":0.71999999999999997},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.28899999999999998},"max":{"1min":7.1310000000000002,"5min":8.0440000000000005,"15min":12.478999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.79800000000000004,"15min":0.73699999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.30599999999999999},"max":{"1min":0.86899999999999999,"5min":7.9409999999999998,"15min":12.391999999999999},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.0529999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":0.97299999999999998,"5min":20.84,"15min":20.84},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69199999999999995,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.0329999999999999,"5min":4.7240000000000002,"15min":11.762},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.69699999999999995,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.79800000000000004,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.78000000000000003,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.9590000000000001,"5min":13.177,"15min":13.177},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.79400000000000004,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.35099999999999998,"15min":0.32600000000000001},"max":{"1min":1.8420000000000001,"5min":6.2309999999999999,"15min":11.865},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1031156,"kb_used_data":57452,"kb_used_omap":1,"kb_used_meta":973694,"kb_avail":93340684,"statfs":{"total":96636764160,"available":95580860416,"internally_reserved":0,"allocated":58830848,"data_stored":58655239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":997062923},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49399999999999999,"15min":0.439},"min":{"1min":0.19400000000000001,"5min":0.186,"15min":0.184},"max":{"1min":0.89000000000000001,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.219,"5min":0.219,"15min":0.19700000000000001},"max":{"1min":2.145,"5min":6.5339999999999998,"15min":9.2249999999999996},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54800000000000004,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":2.161,"5min":13.553000000000001,"15min":13.553000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":2.0910000000000002,"5min":6.6050000000000004,"15min":7.1180000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.70699999999999996,"15min":0.64000000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.22900000000000001},"max":{"1min":2.1190000000000002,"5min":10.236000000000001,"15min":24.879000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.53400000000000003},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.22900000000000001},"max":{"1min":1.4670000000000001,"5min":2.2290000000000001,"15min":20.995000000000001},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":7.1609999999999996,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":8.4420000000000002,"5min":8.4420000000000002,"15min":8.4420000000000002},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9059999999999999,"5min":3.5800000000000001,"15min":8.6419999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.79100000000000004,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.96699999999999997,"5min":16.943999999999999,"15min":16.943999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.0270000000000001,"5min":3.5409999999999999,"15min":7.468},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1,"5min":3.4950000000000001,"15min":5.5030000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":2.5169999999999999,"5min":8.593,"15min":9.3209999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.40200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.496,"5min":6.7859999999999996,"15min":10.346},"last":0.80400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":871104,"kb_used_data":49976,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":93500736,"statfs":{"total":96636764160,"available":95744753664,"internally_reserved":0,"allocated":51175424,"data_stored":51009794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":840825099},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.45500000000000002},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.17499999999999999},"max":{"1min":2.8149999999999999,"5min":12.734,"15min":12.734},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52000000000000002,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":3.976,"5min":9.3620000000000001,"15min":9.3620000000000001},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.843,"5min":15.375,"15min":15.375},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.48099999999999998},"min":{"1min":0.28399999999999997,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.008,"5min":9.2959999999999994,"15min":9.2959999999999994},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.20699999999999999},"max":{"1min":2.7040000000000002,"5min":8.7230000000000008,"15min":8.7230000000000008},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":2.9169999999999998,"5min":10.928000000000001,"15min":12.864000000000001},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0350000000000001,"5min":9.2319999999999993,"15min":9.2319999999999993},"last":0.435},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":3.7930000000000001,"15min":7.5499999999999998},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.61899999999999999},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":3.0790000000000002,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.68700000000000006},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"max":{"1min":2.8660000000000001,"5min":10.135999999999999,"15min":11.068},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.69699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.8959999999999999,"5min":12.77,"15min":12.77},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":4.0579999999999998,"5min":9.391,"15min":9.391},"last":0.375}]},{"osd":7,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73299999999999998,"15min":0.67900000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.0830000000000002,"5min":9.3290000000000006,"15min":9.3290000000000006},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.71499999999999997,"15min":0.65400000000000003},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.33300000000000002},"max":{"1min":0.78900000000000003,"5min":7.8739999999999997,"15min":7.8739999999999997},"last":0.47399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58568704,"data_stored":58559811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45989888,"data_stored":45989473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43065344,"data_stored":43049741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":71307264,"data_stored":71303258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":71311360,"data_stored":71303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114360320,"data_stored":114353365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T22:05:26.694 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T22:05:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.836+0000 7fabe34d5640 1 Processor -- start 2024-03-14T22:05:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.836+0000 7fabe34d5640 1 -- start start 2024-03-14T22:05:26.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe34d5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fabdc148350 0x7fabdc1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe34d5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc06d670 con 0x7fabdc06cf10 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe34d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc0b8c40 con 0x7fabdc0b9930 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe34d5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc0b8f20 con 0x7fabdc148350 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe1a72640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fabdc148350 0x7fabdc1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe1a72640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fabdc148350 0x7fabdc1530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50118/0 (socket says 172.21.15.49:50118) 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe1271640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fabdc06cf10 0x7fabdc147c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43560/0 (socket says 172.21.15.49:43560) 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.837+0000 7fabe1a72640 1 -- 172.21.15.49:0/1481849224 learned_addr learned my addr 172.21.15.49:0/1481849224 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.838+0000 7fabe1a72640 1 -- 172.21.15.49:0/1481849224 >> v1:172.21.15.97:6789/0 conn(0x7fabdc0b9930 legacy=0x7fabdc1415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.838+0000 7fabe1a72640 1 -- 172.21.15.49:0/1481849224 >> v1:172.21.15.49:6789/0 conn(0x7fabdc06cf10 legacy=0x7fabdc147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.838+0000 7fabe1a72640 1 -- 172.21.15.49:0/1481849224 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0b9200 con 0x7fabdc148350 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.838+0000 7fabe1a72640 1 --2- 172.21.15.49:0/1481849224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc148350 0x7fabdc1530c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009870 tx=0x7fabd802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89acbfe7b225482e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.838+0000 7fabd3fff640 1 -- 172.21.15.49:0/1481849224 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd802f840 con 0x7fabdc148350 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.838+0000 7fabd3fff640 1 -- 172.21.15.49:0/1481849224 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd802f9a0 con 0x7fabdc148350 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.838+0000 7fabd3fff640 1 -- 172.21.15.49:0/1481849224 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd802fd30 con 0x7fabdc148350 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.839+0000 7fabe34d5640 1 -- 172.21.15.49:0/1481849224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc148350 msgr2=0x7fabdc1530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.839+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1481849224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc148350 0x7fabdc1530c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009870 tx=0x7fabd802ba20 comp rx=0 tx=0).stop 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.839+0000 7fabe34d5640 1 -- 172.21.15.49:0/1481849224 shutdown_connections 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.839+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1481849224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc148350 0x7fabdc1530c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.839+0000 7fabe34d5640 1 -- 172.21.15.49:0/1481849224 >> 172.21.15.49:0/1481849224 conn(0x7fabdc0b9d30 msgr2=0x7fabdc0ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.839+0000 7fabe34d5640 1 -- 172.21.15.49:0/1481849224 shutdown_connections 2024-03-14T22:05:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.839+0000 7fabe34d5640 1 -- 172.21.15.49:0/1481849224 wait complete. 2024-03-14T22:05:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.840+0000 7fabe34d5640 1 Processor -- start 2024-03-14T22:05:26.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.840+0000 7fabe34d5640 1 -- start start 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe34d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc06cf10 0x7fabdc144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe34d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabdc0b9930 0x7fabdc0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe34d5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabdc148350 0x7fabdc069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe34d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc06a490 con 0x7fabdc06cf10 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe34d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc1e0e50 con 0x7fabdc0b9930 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe34d5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0b7170 con 0x7fabdc148350 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc06cf10 0x7fabdc144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc06cf10 0x7fabdc144d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50128/0 (socket says 172.21.15.49:50128) 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 -- 172.21.15.49:0/1646057940 learned_addr learned my addr 172.21.15.49:0/1646057940 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe0a70640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabdc0b9930 0x7fabdc0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1a72640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabdc148350 0x7fabdc069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 -- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabdc148350 msgr2=0x7fabdc069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabdc148350 0x7fabdc069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 -- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabdc0b9930 msgr2=0x7fabdc0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabdc0b9930 0x7fabdc0b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 -- 172.21.15.49:0/1646057940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd80094d0 con 0x7fabdc06cf10 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabe1271640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc06cf10 0x7fabdc144d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c7a0 tx=0x7fabd400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.841+0000 7fabd1ffb640 1 -- 172.21.15.49:0/1646057940 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400a640 con 0x7fabdc06cf10 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.842+0000 7fabd1ffb640 1 -- 172.21.15.49:0/1646057940 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd400a7a0 con 0x7fabdc06cf10 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.842+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc1e10a0 con 0x7fabdc06cf10 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.842+0000 7fabd1ffb640 1 -- 172.21.15.49:0/1646057940 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400ab10 con 0x7fabdc06cf10 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.842+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc1e1660 con 0x7fabdc06cf10 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.843+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc1e1880 con 0x7fabdc06cf10 2024-03-14T22:05:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.843+0000 7fabd1ffb640 1 -- 172.21.15.49:0/1646057940 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fabd400ac70 con 0x7fabdc06cf10 2024-03-14T22:05:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.843+0000 7fabd1ffb640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fabc006a7a0 0x7fabc006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.844+0000 7fabd1ffb640 1 -- 172.21.15.49:0/1646057940 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fabd40884d0 con 0x7fabdc06cf10 2024-03-14T22:05:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.844+0000 7fabe0a70640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fabc006a7a0 0x7fabc006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.844+0000 7fabe0a70640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fabc006a7a0 0x7fabc006cc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fabdc147430 tx=0x7fabcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:26.848+0000 7fabd1ffb640 1 -- 172.21.15.49:0/1646057940 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fabd4053930 con 0x7fabdc06cf10 2024-03-14T22:05:27.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.023+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabdc06d2f0 con 0x7fabc006a7a0 2024-03-14T22:05:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.031+0000 7fabd1ffb640 1 -- 172.21.15.49:0/1646057940 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292078 (secure 0 0 0) 0x7fabcc001d10 con 0x7fabc006a7a0 2024-03-14T22:05:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:05:27.034 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fabc006a7a0 msgr2=0x7fabc006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fabc006a7a0 0x7fabc006cc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fabdc147430 tx=0x7fabcc009040 comp rx=0 tx=0).stop 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc06cf10 msgr2=0x7fabdc144d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc06cf10 0x7fabdc144d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c7a0 tx=0x7fabd400cc60 comp rx=0 tx=0).stop 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 shutdown_connections 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fabc006a7a0 0x7fabc006cc60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabdc148350 0x7fabdc069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabdc0b9930 0x7fabdc0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 --2- 172.21.15.49:0/1646057940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabdc06cf10 0x7fabdc144d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 >> 172.21.15.49:0/1646057940 conn(0x7fabdc0b9d30 msgr2=0x7fabdc0a10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 shutdown_connections 2024-03-14T22:05:27.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.034+0000 7fabe34d5640 1 -- 172.21.15.49:0/1646057940 wait complete. 2024-03-14T22:05:27.044 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":1564,"stamp":"2024-03-14T22:05:25.588581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257405714,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":4401,"num_read_kb":112072,"num_write":843853,"num_write_kb":9407839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896061,"ondisk_log_size":896061,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8147260,"kb_used_data":504700,"kb_used_omap":13,"kb_used_meta":7642482,"kb_avail":746827460,"statfs":{"total":773094113280,"available":764751319040,"internally_reserved":0,"allocated":516812800,"data_stored":515502560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":7825901656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11540782,"num_objects":-2092,"num_object_clones":0,"num_object_copies":-4184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2092,"num_whiteouts":0,"num_read":14,"num_read_kb":226,"num_write":2321,"num_write_kb":8782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006146"},"pg_stats":[{"pgid":"3.3f","version":"23'10328","reported_seq":10370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.391677+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T22:05:17.391677+0000","last_peered":"2024-03-14T22:05:17.391677+0000","last_clean":"2024-03-14T22:05:17.391677+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T22:05:17.391677+0000","last_undegraded":"2024-03-14T22:05:17.391677+0000","last_fullsized":"2024-03-14T22:05:17.391677+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:57:09.325211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8785,"num_write_kb":106962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'9093","reported_seq":9142,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.446982+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T22:05:17.446982+0000","last_peered":"2024-03-14T22:05:17.446982+0000","last_clean":"2024-03-14T22:05:17.446982+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T22:05:17.446982+0000","last_undegraded":"2024-03-14T22:05:17.446982+0000","last_fullsized":"2024-03-14T22:05:17.446982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9093,"log_dups_size":0,"ondisk_log_size":9093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":7584,"num_write_kb":103945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'11562","reported_seq":11603,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.356641+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T22:05:23.356641+0000","last_peered":"2024-03-14T22:05:23.356641+0000","last_clean":"2024-03-14T22:05:23.356641+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T22:05:23.356641+0000","last_undegraded":"2024-03-14T22:05:23.356641+0000","last_fullsized":"2024-03-14T22:05:23.356641+0000","mapping_epoch":19,"log_start":"23'1500","ondisk_log_start":"23'1500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:08:55.492956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1274,"num_write":9914,"num_write_kb":127294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'10095","reported_seq":10141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.609893+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T22:05:16.609893+0000","last_peered":"2024-03-14T22:05:16.609893+0000","last_clean":"2024-03-14T22:05:16.609893+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T22:05:16.609893+0000","last_undegraded":"2024-03-14T22:05:16.609893+0000","last_fullsized":"2024-03-14T22:05:16.609893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8407,"num_write_kb":110879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'9604","reported_seq":9638,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.547040+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T22:05:16.547040+0000","last_peered":"2024-03-14T22:05:16.547040+0000","last_clean":"2024-03-14T22:05:16.547040+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T22:05:16.547040+0000","last_undegraded":"2024-03-14T22:05:16.547040+0000","last_fullsized":"2024-03-14T22:05:16.547040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9604,"log_dups_size":0,"ondisk_log_size":9604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8008,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'9639","reported_seq":9679,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.400588+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T22:05:17.400588+0000","last_peered":"2024-03-14T22:05:17.400588+0000","last_clean":"2024-03-14T22:05:17.400588+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T22:05:17.400588+0000","last_undegraded":"2024-03-14T22:05:17.400588+0000","last_fullsized":"2024-03-14T22:05:17.400588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":8054,"num_write_kb":114907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10032","reported_seq":10068,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431473+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T22:05:17.431473+0000","last_peered":"2024-03-14T22:05:17.431473+0000","last_clean":"2024-03-14T22:05:17.431473+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T22:05:17.431473+0000","last_undegraded":"2024-03-14T22:05:17.431473+0000","last_fullsized":"2024-03-14T22:05:17.431473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":8423,"num_write_kb":104789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'11190","reported_seq":11230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.614200+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T22:05:16.614200+0000","last_peered":"2024-03-14T22:05:16.614200+0000","last_clean":"2024-03-14T22:05:16.614200+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T22:05:16.614200+0000","last_undegraded":"2024-03-14T22:05:16.614200+0000","last_fullsized":"2024-03-14T22:05:16.614200+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":9605,"num_write_kb":125652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'9011","reported_seq":9048,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.436028+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T22:05:17.436028+0000","last_peered":"2024-03-14T22:05:17.436028+0000","last_clean":"2024-03-14T22:05:17.436028+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T22:05:17.436028+0000","last_undegraded":"2024-03-14T22:05:17.436028+0000","last_fullsized":"2024-03-14T22:05:17.436028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9011,"log_dups_size":0,"ondisk_log_size":9011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7381,"num_write_kb":104568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'9532","reported_seq":9568,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.411246+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T22:05:17.411246+0000","last_peered":"2024-03-14T22:05:17.411246+0000","last_clean":"2024-03-14T22:05:17.411246+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T22:05:17.411246+0000","last_undegraded":"2024-03-14T22:05:17.411246+0000","last_fullsized":"2024-03-14T22:05:17.411246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7843,"num_write_kb":113016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'9221","reported_seq":9262,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.399007+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T22:05:17.399007+0000","last_peered":"2024-03-14T22:05:17.399007+0000","last_clean":"2024-03-14T22:05:17.399007+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T22:05:17.399007+0000","last_undegraded":"2024-03-14T22:05:17.399007+0000","last_fullsized":"2024-03-14T22:05:17.399007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7677,"num_write_kb":104837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'10300","reported_seq":10335,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431722+0000","last_change":"2024-03-14T21:14:03.034043+0000","last_active":"2024-03-14T22:05:17.431722+0000","last_peered":"2024-03-14T22:05:17.431722+0000","last_clean":"2024-03-14T22:05:17.431722+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T22:05:17.431722+0000","last_undegraded":"2024-03-14T22:05:17.431722+0000","last_fullsized":"2024-03-14T22:05:17.431722+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:33.787920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8730,"num_write_kb":105734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'8860","reported_seq":8898,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.540388+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T22:05:16.540388+0000","last_peered":"2024-03-14T22:05:16.540388+0000","last_clean":"2024-03-14T22:05:16.540388+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T22:05:16.540388+0000","last_undegraded":"2024-03-14T22:05:16.540388+0000","last_fullsized":"2024-03-14T22:05:16.540388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8860,"log_dups_size":0,"ondisk_log_size":8860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":7264,"num_write_kb":95419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'10404","reported_seq":10443,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.392084+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T22:05:17.392084+0000","last_peered":"2024-03-14T22:05:17.392084+0000","last_clean":"2024-03-14T22:05:17.392084+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T22:05:17.392084+0000","last_undegraded":"2024-03-14T22:05:17.392084+0000","last_fullsized":"2024-03-14T22:05:17.392084+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8793,"num_write_kb":111405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'9697","reported_seq":9734,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.610064+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T22:05:16.610064+0000","last_peered":"2024-03-14T22:05:16.610064+0000","last_clean":"2024-03-14T22:05:16.610064+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T22:05:16.610064+0000","last_undegraded":"2024-03-14T22:05:16.610064+0000","last_fullsized":"2024-03-14T22:05:16.610064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9697,"log_dups_size":0,"ondisk_log_size":9697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8012,"num_write_kb":95820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'9656","reported_seq":9704,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.455534+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T22:05:17.455534+0000","last_peered":"2024-03-14T22:05:17.455534+0000","last_clean":"2024-03-14T22:05:17.455534+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T22:05:17.455534+0000","last_undegraded":"2024-03-14T22:05:17.455534+0000","last_fullsized":"2024-03-14T22:05:17.455534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9656,"log_dups_size":0,"ondisk_log_size":9656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3572,"num_write":8076,"num_write_kb":107857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8658","reported_seq":8695,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.457733+0000","last_change":"2024-03-14T21:14:44.859268+0000","last_active":"2024-03-14T22:05:17.457733+0000","last_peered":"2024-03-14T22:05:17.457733+0000","last_clean":"2024-03-14T22:05:17.457733+0000","last_became_active":"2024-03-14T21:14:44.858906+0000","last_became_peered":"2024-03-14T21:14:44.858906+0000","last_unstale":"2024-03-14T22:05:17.457733+0000","last_undegraded":"2024-03-14T22:05:17.457733+0000","last_fullsized":"2024-03-14T22:05:17.457733+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8658,"log_dups_size":0,"ondisk_log_size":8658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:56:07.747779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1528,"num_write":7020,"num_write_kb":102847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10194","reported_seq":10230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.574269+0000","last_change":"2024-03-14T21:14:03.029823+0000","last_active":"2024-03-14T22:05:16.574269+0000","last_peered":"2024-03-14T22:05:16.574269+0000","last_clean":"2024-03-14T22:05:16.574269+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T22:05:16.574269+0000","last_undegraded":"2024-03-14T22:05:16.574269+0000","last_fullsized":"2024-03-14T22:05:16.574269+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:47.131679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8552,"num_write_kb":105487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'10515","reported_seq":10549,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.365482+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T22:05:23.365482+0000","last_peered":"2024-03-14T22:05:23.365482+0000","last_clean":"2024-03-14T22:05:23.365482+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T22:05:23.365482+0000","last_undegraded":"2024-03-14T22:05:23.365482+0000","last_fullsized":"2024-03-14T22:05:23.365482+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":118219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'10892","reported_seq":10930,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.430759+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T22:05:17.430759+0000","last_peered":"2024-03-14T22:05:17.430759+0000","last_clean":"2024-03-14T22:05:17.430759+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T22:05:17.430759+0000","last_undegraded":"2024-03-14T22:05:17.430759+0000","last_fullsized":"2024-03-14T22:05:17.430759+0000","mapping_epoch":19,"log_start":"23'800","ondisk_log_start":"23'800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":9315,"num_write_kb":117297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'10951","reported_seq":10997,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.442792+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T22:05:17.442792+0000","last_peered":"2024-03-14T22:05:17.442792+0000","last_clean":"2024-03-14T22:05:17.442792+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T22:05:17.442792+0000","last_undegraded":"2024-03-14T22:05:17.442792+0000","last_fullsized":"2024-03-14T22:05:17.442792+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3050,"num_write":9338,"num_write_kb":109552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10716","reported_seq":10752,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.349639+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T22:05:23.349639+0000","last_peered":"2024-03-14T22:05:23.349639+0000","last_clean":"2024-03-14T22:05:23.349639+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T22:05:23.349639+0000","last_undegraded":"2024-03-14T22:05:23.349639+0000","last_fullsized":"2024-03-14T22:05:23.349639+0000","mapping_epoch":19,"log_start":"23'700","ondisk_log_start":"23'700","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":9066,"num_write_kb":118743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'10557","reported_seq":10593,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.383547+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T22:05:17.383547+0000","last_peered":"2024-03-14T22:05:17.383547+0000","last_clean":"2024-03-14T22:05:17.383547+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T22:05:17.383547+0000","last_undegraded":"2024-03-14T22:05:17.383547+0000","last_fullsized":"2024-03-14T22:05:17.383547+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8950,"num_write_kb":109690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'11172","reported_seq":11213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.420018+0000","last_change":"2024-03-14T21:14:03.038694+0000","last_active":"2024-03-14T22:05:17.420018+0000","last_peered":"2024-03-14T22:05:17.420018+0000","last_clean":"2024-03-14T22:05:17.420018+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T22:05:17.420018+0000","last_undegraded":"2024-03-14T22:05:17.420018+0000","last_fullsized":"2024-03-14T22:05:17.420018+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1530,"num_write":9571,"num_write_kb":127549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'10188","reported_seq":10225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.442741+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T22:05:17.442741+0000","last_peered":"2024-03-14T22:05:17.442741+0000","last_clean":"2024-03-14T22:05:17.442741+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T22:05:17.442741+0000","last_undegraded":"2024-03-14T22:05:17.442741+0000","last_fullsized":"2024-03-14T22:05:17.442741+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":8582,"num_write_kb":110246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'12329","reported_seq":12370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.390032+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T22:05:17.390032+0000","last_peered":"2024-03-14T22:05:17.390032+0000","last_clean":"2024-03-14T22:05:17.390032+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T22:05:17.390032+0000","last_undegraded":"2024-03-14T22:05:17.390032+0000","last_fullsized":"2024-03-14T22:05:17.390032+0000","mapping_epoch":19,"log_start":"23'2300","ondisk_log_start":"23'2300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10697,"num_write_kb":121303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'9238","reported_seq":9281,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.393718+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T22:05:17.393718+0000","last_peered":"2024-03-14T22:05:17.393718+0000","last_clean":"2024-03-14T22:05:17.393718+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T22:05:17.393718+0000","last_undegraded":"2024-03-14T22:05:17.393718+0000","last_fullsized":"2024-03-14T22:05:17.393718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9238,"log_dups_size":0,"ondisk_log_size":9238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2628,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7599,"num_write_kb":100407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'10304","reported_seq":10339,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.430400+0000","last_change":"2024-03-14T21:14:03.040690+0000","last_active":"2024-03-14T22:05:17.430400+0000","last_peered":"2024-03-14T22:05:17.430400+0000","last_clean":"2024-03-14T22:05:17.430400+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T22:05:17.430400+0000","last_undegraded":"2024-03-14T22:05:17.430400+0000","last_fullsized":"2024-03-14T22:05:17.430400+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":108137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'11950","reported_seq":11998,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.459036+0000","last_change":"2024-03-14T21:14:44.746590+0000","last_active":"2024-03-14T22:05:17.459036+0000","last_peered":"2024-03-14T22:05:17.459036+0000","last_clean":"2024-03-14T22:05:17.459036+0000","last_became_active":"2024-03-14T21:14:44.746271+0000","last_became_peered":"2024-03-14T21:14:44.746271+0000","last_unstale":"2024-03-14T22:05:17.459036+0000","last_undegraded":"2024-03-14T22:05:17.459036+0000","last_fullsized":"2024-03-14T22:05:17.459036+0000","mapping_epoch":22,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:01:46.971606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":10306,"num_write_kb":120667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'11949","reported_seq":11991,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431315+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T22:05:17.431315+0000","last_peered":"2024-03-14T22:05:17.431315+0000","last_clean":"2024-03-14T22:05:17.431315+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T22:05:17.431315+0000","last_undegraded":"2024-03-14T22:05:17.431315+0000","last_fullsized":"2024-03-14T22:05:17.431315+0000","mapping_epoch":19,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10416,"num_write_kb":126678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'11288","reported_seq":11335,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.451854+0000","last_change":"2024-03-14T21:14:03.029847+0000","last_active":"2024-03-14T22:05:17.451854+0000","last_peered":"2024-03-14T22:05:17.451854+0000","last_clean":"2024-03-14T22:05:17.451854+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T22:05:17.451854+0000","last_undegraded":"2024-03-14T22:05:17.451854+0000","last_fullsized":"2024-03-14T22:05:17.451854+0000","mapping_epoch":19,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2806,"num_write":9622,"num_write_kb":118224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'8631","reported_seq":8668,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.386380+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T22:05:17.386380+0000","last_peered":"2024-03-14T22:05:17.386380+0000","last_clean":"2024-03-14T22:05:17.386380+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T22:05:17.386380+0000","last_undegraded":"2024-03-14T22:05:17.386380+0000","last_fullsized":"2024-03-14T22:05:17.386380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":7051,"num_write_kb":105885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'9047","reported_seq":9083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431703+0000","last_change":"2024-03-14T21:14:03.037915+0000","last_active":"2024-03-14T22:05:17.431703+0000","last_peered":"2024-03-14T22:05:17.431703+0000","last_clean":"2024-03-14T22:05:17.431703+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T22:05:17.431703+0000","last_undegraded":"2024-03-14T22:05:17.431703+0000","last_fullsized":"2024-03-14T22:05:17.431703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":7364,"num_write_kb":104642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'10861","reported_seq":10894,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422940+0000","last_change":"2024-03-14T21:14:02.655762+0000","last_active":"2024-03-14T22:05:23.422940+0000","last_peered":"2024-03-14T22:05:23.422940+0000","last_clean":"2024-03-14T22:05:23.422940+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T22:05:23.422940+0000","last_undegraded":"2024-03-14T22:05:23.422940+0000","last_fullsized":"2024-03-14T22:05:23.422940+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10855,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'9532","reported_seq":9571,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.556727+0000","last_change":"2024-03-14T21:14:03.033091+0000","last_active":"2024-03-14T22:05:16.556727+0000","last_peered":"2024-03-14T22:05:16.556727+0000","last_clean":"2024-03-14T22:05:16.556727+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T22:05:16.556727+0000","last_undegraded":"2024-03-14T22:05:16.556727+0000","last_fullsized":"2024-03-14T22:05:16.556727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":7923,"num_write_kb":107722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10915","reported_seq":10948,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.394449+0000","last_change":"2024-03-14T21:14:02.662899+0000","last_active":"2024-03-14T22:05:23.394449+0000","last_peered":"2024-03-14T22:05:23.394449+0000","last_clean":"2024-03-14T22:05:23.394449+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T22:05:23.394449+0000","last_undegraded":"2024-03-14T22:05:23.394449+0000","last_fullsized":"2024-03-14T22:05:23.394449+0000","mapping_epoch":18,"log_start":"23'900","ondisk_log_start":"23'900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":10937,"num_write_kb":77181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'7036","reported_seq":7070,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422556+0000","last_change":"2024-03-14T21:14:02.660574+0000","last_active":"2024-03-14T22:05:23.422556+0000","last_peered":"2024-03-14T22:05:23.422556+0000","last_clean":"2024-03-14T22:05:23.422556+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T22:05:23.422556+0000","last_undegraded":"2024-03-14T22:05:23.422556+0000","last_fullsized":"2024-03-14T22:05:23.422556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":7036,"log_dups_size":0,"ondisk_log_size":7036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7046,"num_write_kb":58697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'11484","reported_seq":11518,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.365328+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T22:05:23.365328+0000","last_peered":"2024-03-14T22:05:23.365328+0000","last_clean":"2024-03-14T22:05:23.365328+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T22:05:23.365328+0000","last_undegraded":"2024-03-14T22:05:23.365328+0000","last_fullsized":"2024-03-14T22:05:23.365328+0000","mapping_epoch":19,"log_start":"23'1400","ondisk_log_start":"23'1400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":9890,"num_write_kb":106254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'11076","reported_seq":11110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.352182+0000","last_change":"2024-03-14T21:14:02.661303+0000","last_active":"2024-03-14T22:05:17.352182+0000","last_peered":"2024-03-14T22:05:17.352182+0000","last_clean":"2024-03-14T22:05:17.352182+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T22:05:17.352182+0000","last_undegraded":"2024-03-14T22:05:17.352182+0000","last_fullsized":"2024-03-14T22:05:17.352182+0000","mapping_epoch":18,"log_start":"23'1000","ondisk_log_start":"23'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11078,"num_write_kb":81064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'10471","reported_seq":10515,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.565713+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T22:05:16.565713+0000","last_peered":"2024-03-14T22:05:16.565713+0000","last_clean":"2024-03-14T22:05:16.565713+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T22:05:16.565713+0000","last_undegraded":"2024-03-14T22:05:16.565713+0000","last_fullsized":"2024-03-14T22:05:16.565713+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2800,"num_write":8909,"num_write_kb":114509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8286","reported_seq":8925,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.553161+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T22:05:15.553161+0000","last_peered":"2024-03-14T22:05:15.553161+0000","last_clean":"2024-03-14T22:05:15.553161+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T22:05:15.553161+0000","last_undegraded":"2024-03-14T22:05:15.553161+0000","last_fullsized":"2024-03-14T22:05:15.553161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8286,"log_dups_size":0,"ondisk_log_size":8286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":607,"num_read_kb":4494,"num_write":8292,"num_write_kb":66794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10197","reported_seq":10235,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.586675+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T22:05:16.586675+0000","last_peered":"2024-03-14T22:05:16.586675+0000","last_clean":"2024-03-14T22:05:16.586675+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T22:05:16.586675+0000","last_undegraded":"2024-03-14T22:05:16.586675+0000","last_fullsized":"2024-03-14T22:05:16.586675+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8609,"num_write_kb":104201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'10690","reported_seq":10724,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.418804+0000","last_change":"2024-03-14T21:14:02.660384+0000","last_active":"2024-03-14T22:05:23.418804+0000","last_peered":"2024-03-14T22:05:23.418804+0000","last_clean":"2024-03-14T22:05:23.418804+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T22:05:23.418804+0000","last_undegraded":"2024-03-14T22:05:23.418804+0000","last_fullsized":"2024-03-14T22:05:23.418804+0000","mapping_epoch":18,"log_start":"23'600","ondisk_log_start":"23'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":92935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'9094","reported_seq":9131,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.421535+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T22:05:17.421535+0000","last_peered":"2024-03-14T22:05:17.421535+0000","last_clean":"2024-03-14T22:05:17.421535+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T22:05:17.421535+0000","last_undegraded":"2024-03-14T22:05:17.421535+0000","last_fullsized":"2024-03-14T22:05:17.421535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9094,"log_dups_size":0,"ondisk_log_size":9094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:57:33.155885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7455,"num_write_kb":104328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9143","reported_seq":9177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.618955+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T22:05:16.618955+0000","last_peered":"2024-03-14T22:05:16.618955+0000","last_clean":"2024-03-14T22:05:16.618955+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T22:05:16.618955+0000","last_undegraded":"2024-03-14T22:05:16.618955+0000","last_fullsized":"2024-03-14T22:05:16.618955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9156,"num_write_kb":60300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11326","reported_seq":11369,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.356317+0000","last_change":"2024-03-14T21:14:03.032533+0000","last_active":"2024-03-14T22:05:23.356317+0000","last_peered":"2024-03-14T22:05:23.356317+0000","last_clean":"2024-03-14T22:05:23.356317+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T22:05:23.356317+0000","last_undegraded":"2024-03-14T22:05:23.356317+0000","last_fullsized":"2024-03-14T22:05:23.356317+0000","mapping_epoch":19,"log_start":"23'1300","ondisk_log_start":"23'1300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:41.581386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2542,"num_write":9699,"num_write_kb":113170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'13351","reported_seq":13939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.410668+0000","last_change":"2024-03-14T21:14:02.659028+0000","last_active":"2024-03-14T22:05:23.410668+0000","last_peered":"2024-03-14T22:05:23.410668+0000","last_clean":"2024-03-14T22:05:23.410668+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T22:05:23.410668+0000","last_undegraded":"2024-03-14T22:05:23.410668+0000","last_fullsized":"2024-03-14T22:05:23.410668+0000","mapping_epoch":18,"log_start":"23'3300","ondisk_log_start":"23'3300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":557,"num_read_kb":4455,"num_write":13349,"num_write_kb":107014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'9102","reported_seq":9148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.342305+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T22:05:17.342305+0000","last_peered":"2024-03-14T22:05:17.342305+0000","last_clean":"2024-03-14T22:05:17.342305+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T22:05:17.342305+0000","last_undegraded":"2024-03-14T22:05:17.342305+0000","last_fullsized":"2024-03-14T22:05:17.342305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9102,"log_dups_size":0,"ondisk_log_size":9102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3318,"num_write":7536,"num_write_kb":95248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'8009","reported_seq":8043,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.437600+0000","last_change":"2024-03-14T21:14:02.663139+0000","last_active":"2024-03-14T22:05:23.437600+0000","last_peered":"2024-03-14T22:05:23.437600+0000","last_clean":"2024-03-14T22:05:23.437600+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T22:05:23.437600+0000","last_undegraded":"2024-03-14T22:05:23.437600+0000","last_fullsized":"2024-03-14T22:05:23.437600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8009,"log_dups_size":0,"ondisk_log_size":8009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:25:31.095452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8008,"num_write_kb":62700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'9642","reported_seq":9681,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.460002+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T22:05:17.460002+0000","last_peered":"2024-03-14T22:05:17.460002+0000","last_clean":"2024-03-14T22:05:17.460002+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T22:05:17.460002+0000","last_undegraded":"2024-03-14T22:05:17.460002+0000","last_fullsized":"2024-03-14T22:05:17.460002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9642,"log_dups_size":0,"ondisk_log_size":9642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8033,"num_write_kb":100651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5756","reported_seq":5791,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.375166+0000","last_change":"2024-03-14T21:14:02.659773+0000","last_active":"2024-03-14T22:05:23.375166+0000","last_peered":"2024-03-14T22:05:23.375166+0000","last_clean":"2024-03-14T22:05:23.375166+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T22:05:23.375166+0000","last_undegraded":"2024-03-14T22:05:23.375166+0000","last_fullsized":"2024-03-14T22:05:23.375166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5756,"log_dups_size":0,"ondisk_log_size":5756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5754,"num_write_kb":48600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'9761","reported_seq":9797,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.410634+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T22:05:17.410634+0000","last_peered":"2024-03-14T22:05:17.410634+0000","last_clean":"2024-03-14T22:05:17.410634+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T22:05:17.410634+0000","last_undegraded":"2024-03-14T22:05:17.410634+0000","last_fullsized":"2024-03-14T22:05:17.410634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8135,"num_write_kb":106503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5851","reported_seq":5886,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.412820+0000","last_change":"2024-03-14T21:14:02.663585+0000","last_active":"2024-03-14T22:05:23.412820+0000","last_peered":"2024-03-14T22:05:23.412820+0000","last_clean":"2024-03-14T22:05:23.412820+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T22:05:23.412820+0000","last_undegraded":"2024-03-14T22:05:23.412820+0000","last_fullsized":"2024-03-14T22:05:23.412820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5851,"log_dups_size":0,"ondisk_log_size":5851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":52920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'9078","reported_seq":9122,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.334095+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T22:05:17.334095+0000","last_peered":"2024-03-14T22:05:17.334095+0000","last_clean":"2024-03-14T22:05:17.334095+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T22:05:17.334095+0000","last_undegraded":"2024-03-14T22:05:17.334095+0000","last_fullsized":"2024-03-14T22:05:17.334095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9078,"log_dups_size":0,"ondisk_log_size":9078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2548,"num_write":7495,"num_write_kb":101746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8321","reported_seq":8356,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.432570+0000","last_change":"2024-03-14T21:14:02.654931+0000","last_active":"2024-03-14T22:05:23.432570+0000","last_peered":"2024-03-14T22:05:23.432570+0000","last_clean":"2024-03-14T22:05:23.432570+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T22:05:23.432570+0000","last_undegraded":"2024-03-14T22:05:23.432570+0000","last_fullsized":"2024-03-14T22:05:23.432570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629345,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":8334,"num_write_kb":67488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'10125","reported_seq":10160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.361153+0000","last_change":"2024-03-14T21:14:44.859568+0000","last_active":"2024-03-14T22:05:17.361153+0000","last_peered":"2024-03-14T22:05:17.361153+0000","last_clean":"2024-03-14T22:05:17.361153+0000","last_became_active":"2024-03-14T21:14:44.859185+0000","last_became_peered":"2024-03-14T21:14:44.859185+0000","last_unstale":"2024-03-14T22:05:17.361153+0000","last_undegraded":"2024-03-14T22:05:17.361153+0000","last_fullsized":"2024-03-14T22:05:17.361153+0000","mapping_epoch":22,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:20:25.220662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":8491,"num_write_kb":113136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"23'8046","reported_seq":8081,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430049+0000","last_change":"2024-03-14T21:14:02.663747+0000","last_active":"2024-03-14T22:05:23.430049+0000","last_peered":"2024-03-14T22:05:23.430049+0000","last_clean":"2024-03-14T22:05:23.430049+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T22:05:23.430049+0000","last_undegraded":"2024-03-14T22:05:23.430049+0000","last_fullsized":"2024-03-14T22:05:23.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:59:11.091842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8051,"num_write_kb":62569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'9547","reported_seq":9589,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.579367+0000","last_change":"2024-03-14T21:14:03.030867+0000","last_active":"2024-03-14T22:05:16.579367+0000","last_peered":"2024-03-14T22:05:16.579367+0000","last_clean":"2024-03-14T22:05:16.579367+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T22:05:16.579367+0000","last_undegraded":"2024-03-14T22:05:16.579367+0000","last_fullsized":"2024-03-14T22:05:16.579367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2042,"num_write":7926,"num_write_kb":109666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'10889","reported_seq":10924,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.438118+0000","last_change":"2024-03-14T21:14:02.659095+0000","last_active":"2024-03-14T22:05:23.438118+0000","last_peered":"2024-03-14T22:05:23.438118+0000","last_clean":"2024-03-14T22:05:23.438118+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T22:05:23.438118+0000","last_undegraded":"2024-03-14T22:05:23.438118+0000","last_fullsized":"2024-03-14T22:05:23.438118+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10896,"num_write_kb":88926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9010","reported_seq":9049,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.599835+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T22:05:16.599835+0000","last_peered":"2024-03-14T22:05:16.599835+0000","last_clean":"2024-03-14T22:05:16.599835+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T22:05:16.599835+0000","last_undegraded":"2024-03-14T22:05:16.599835+0000","last_fullsized":"2024-03-14T22:05:16.599835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9010,"log_dups_size":0,"ondisk_log_size":9010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":7443,"num_write_kb":93234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'11492","reported_seq":11527,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:03:35.733472+0000","last_change":"2024-03-14T21:14:02.663441+0000","last_active":"2024-03-14T22:03:35.733472+0000","last_peered":"2024-03-14T22:03:35.733472+0000","last_clean":"2024-03-14T22:03:35.733472+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T22:03:35.733472+0000","last_undegraded":"2024-03-14T22:03:35.733472+0000","last_fullsized":"2024-03-14T22:03:35.733472+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11486,"num_write_kb":85212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'9510","reported_seq":9550,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.408549+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T22:05:17.408549+0000","last_peered":"2024-03-14T22:05:17.408549+0000","last_clean":"2024-03-14T22:05:17.408549+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T22:05:17.408549+0000","last_undegraded":"2024-03-14T22:05:17.408549+0000","last_fullsized":"2024-03-14T22:05:17.408549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9510,"log_dups_size":0,"ondisk_log_size":9510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":7940,"num_write_kb":115635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8904","reported_seq":8939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421547+0000","last_change":"2024-03-14T21:14:02.653919+0000","last_active":"2024-03-14T22:05:23.421547+0000","last_peered":"2024-03-14T22:05:23.421547+0000","last_clean":"2024-03-14T22:05:23.421547+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T22:05:23.421547+0000","last_undegraded":"2024-03-14T22:05:23.421547+0000","last_fullsized":"2024-03-14T22:05:23.421547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8904,"log_dups_size":0,"ondisk_log_size":8904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8892,"num_write_kb":67382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'9418","reported_seq":9456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431971+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T22:05:17.431971+0000","last_peered":"2024-03-14T22:05:17.431971+0000","last_clean":"2024-03-14T22:05:17.431971+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T22:05:17.431971+0000","last_undegraded":"2024-03-14T22:05:17.431971+0000","last_fullsized":"2024-03-14T22:05:17.431971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9418,"log_dups_size":0,"ondisk_log_size":9418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7820,"num_write_kb":110543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:02:10.082485+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T22:02:10.082485+0000","last_peered":"2024-03-14T22:02:10.082485+0000","last_clean":"2024-03-14T22:02:10.082485+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T22:02:10.082485+0000","last_undegraded":"2024-03-14T22:02:10.082485+0000","last_fullsized":"2024-03-14T22:02:10.082485+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'8577","reported_seq":8614,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430551+0000","last_change":"2024-03-14T21:14:02.652881+0000","last_active":"2024-03-14T22:05:23.430551+0000","last_peered":"2024-03-14T22:05:23.430551+0000","last_clean":"2024-03-14T22:05:23.430551+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T22:05:23.430551+0000","last_undegraded":"2024-03-14T22:05:23.430551+0000","last_fullsized":"2024-03-14T22:05:23.430551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8577,"log_dups_size":0,"ondisk_log_size":8577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":67724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'11125","reported_seq":11164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.609325+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T22:05:16.609325+0000","last_peered":"2024-03-14T22:05:16.609325+0000","last_clean":"2024-03-14T22:05:16.609325+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T22:05:16.609325+0000","last_undegraded":"2024-03-14T22:05:16.609325+0000","last_fullsized":"2024-03-14T22:05:16.609325+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":9507,"num_write_kb":117178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'8676","reported_seq":8714,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421927+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T22:05:23.421927+0000","last_peered":"2024-03-14T22:05:23.421927+0000","last_clean":"2024-03-14T22:05:23.421927+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T22:05:23.421927+0000","last_undegraded":"2024-03-14T22:05:23.421927+0000","last_fullsized":"2024-03-14T22:05:23.421927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8676,"log_dups_size":0,"ondisk_log_size":8676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:46.126855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8672,"num_write_kb":67194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10241","reported_seq":10279,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.608108+0000","last_change":"2024-03-14T21:14:03.033195+0000","last_active":"2024-03-14T22:05:16.608108+0000","last_peered":"2024-03-14T22:05:16.608108+0000","last_clean":"2024-03-14T22:05:16.608108+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T22:05:16.608108+0000","last_undegraded":"2024-03-14T22:05:16.608108+0000","last_fullsized":"2024-03-14T22:05:16.608108+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8656,"num_write_kb":115905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11303","reported_seq":12007,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.404838+0000","last_change":"2024-03-14T21:14:02.658709+0000","last_active":"2024-03-14T22:05:23.404838+0000","last_peered":"2024-03-14T22:05:23.404838+0000","last_clean":"2024-03-14T22:05:23.404838+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T22:05:23.404838+0000","last_undegraded":"2024-03-14T22:05:23.404838+0000","last_fullsized":"2024-03-14T22:05:23.404838+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":669,"num_read_kb":4516,"num_write":11296,"num_write_kb":89038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'9435","reported_seq":9475,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.585185+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T22:05:16.585185+0000","last_peered":"2024-03-14T22:05:16.585185+0000","last_clean":"2024-03-14T22:05:16.585185+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T22:05:16.585185+0000","last_undegraded":"2024-03-14T22:05:16.585185+0000","last_fullsized":"2024-03-14T22:05:16.585185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9435,"log_dups_size":0,"ondisk_log_size":9435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7837,"num_write_kb":123513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'9296","reported_seq":9333,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.408089+0000","last_change":"2024-03-14T21:14:02.657401+0000","last_active":"2024-03-14T22:05:23.408089+0000","last_peered":"2024-03-14T22:05:23.408089+0000","last_clean":"2024-03-14T22:05:23.408089+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T22:05:23.408089+0000","last_undegraded":"2024-03-14T22:05:23.408089+0000","last_fullsized":"2024-03-14T22:05:23.408089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9294,"num_write_kb":71830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'9543","reported_seq":9580,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.459567+0000","last_change":"2024-03-14T21:14:03.036152+0000","last_active":"2024-03-14T22:05:17.459567+0000","last_peered":"2024-03-14T22:05:17.459567+0000","last_clean":"2024-03-14T22:05:17.459567+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T22:05:17.459567+0000","last_undegraded":"2024-03-14T22:05:17.459567+0000","last_fullsized":"2024-03-14T22:05:17.459567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9543,"log_dups_size":0,"ondisk_log_size":9543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:14.794102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":7936,"num_write_kb":109238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'11129","reported_seq":11166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.404373+0000","last_change":"2024-03-14T21:14:02.653065+0000","last_active":"2024-03-14T22:05:17.404373+0000","last_peered":"2024-03-14T22:05:17.404373+0000","last_clean":"2024-03-14T22:05:17.404373+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T22:05:17.404373+0000","last_undegraded":"2024-03-14T22:05:17.404373+0000","last_fullsized":"2024-03-14T22:05:17.404373+0000","mapping_epoch":18,"log_start":"23'1100","ondisk_log_start":"23'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11129,"num_write_kb":85841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'12164","reported_seq":12200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.438275+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T22:05:17.438275+0000","last_peered":"2024-03-14T22:05:17.438275+0000","last_clean":"2024-03-14T22:05:17.438275+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T22:05:17.438275+0000","last_undegraded":"2024-03-14T22:05:17.438275+0000","last_fullsized":"2024-03-14T22:05:17.438275+0000","mapping_epoch":19,"log_start":"23'2100","ondisk_log_start":"23'2100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10560,"num_write_kb":124816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'9283","reported_seq":9320,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.431083+0000","last_change":"2024-03-14T21:14:02.659612+0000","last_active":"2024-03-14T22:05:23.431083+0000","last_peered":"2024-03-14T22:05:23.431083+0000","last_clean":"2024-03-14T22:05:23.431083+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T22:05:23.431083+0000","last_undegraded":"2024-03-14T22:05:23.431083+0000","last_fullsized":"2024-03-14T22:05:23.431083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9283,"log_dups_size":0,"ondisk_log_size":9283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9294,"num_write_kb":71728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9769","reported_seq":9810,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.417912+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T22:05:17.417912+0000","last_peered":"2024-03-14T22:05:17.417912+0000","last_clean":"2024-03-14T22:05:17.417912+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T22:05:17.417912+0000","last_undegraded":"2024-03-14T22:05:17.417912+0000","last_fullsized":"2024-03-14T22:05:17.417912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9769,"log_dups_size":0,"ondisk_log_size":9769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8091,"num_write_kb":107879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'8352","reported_seq":8981,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:11.741680+0000","last_change":"2024-03-14T21:14:02.662122+0000","last_active":"2024-03-14T22:05:11.741680+0000","last_peered":"2024-03-14T22:05:11.741680+0000","last_clean":"2024-03-14T22:05:11.741680+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T22:05:11.741680+0000","last_undegraded":"2024-03-14T22:05:11.741680+0000","last_fullsized":"2024-03-14T22:05:11.741680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8352,"log_dups_size":0,"ondisk_log_size":8352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":593,"num_read_kb":4491,"num_write":9527,"num_write_kb":73065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'10070","reported_seq":10114,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.350964+0000","last_change":"2024-03-14T21:14:03.037921+0000","last_active":"2024-03-14T22:05:23.350964+0000","last_peered":"2024-03-14T22:05:23.350964+0000","last_clean":"2024-03-14T22:05:23.350964+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T22:05:23.350964+0000","last_undegraded":"2024-03-14T22:05:23.350964+0000","last_fullsized":"2024-03-14T22:05:23.350964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8532,"num_write_kb":114847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5880","reported_seq":5916,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.420296+0000","last_change":"2024-03-14T21:14:02.665745+0000","last_active":"2024-03-14T22:05:23.420296+0000","last_peered":"2024-03-14T22:05:23.420296+0000","last_clean":"2024-03-14T22:05:23.420296+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T22:05:23.420296+0000","last_undegraded":"2024-03-14T22:05:23.420296+0000","last_fullsized":"2024-03-14T22:05:23.420296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5880,"log_dups_size":0,"ondisk_log_size":5880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5885,"num_write_kb":44672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'10911","reported_seq":10953,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.453069+0000","last_change":"2024-03-14T21:14:03.029269+0000","last_active":"2024-03-14T22:05:17.453069+0000","last_peered":"2024-03-14T22:05:17.453069+0000","last_clean":"2024-03-14T22:05:17.453069+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T22:05:17.453069+0000","last_undegraded":"2024-03-14T22:05:17.453069+0000","last_fullsized":"2024-03-14T22:05:17.453069+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":9274,"num_write_kb":112928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'4360","reported_seq":4396,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.551031+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T22:05:15.551031+0000","last_peered":"2024-03-14T22:05:15.551031+0000","last_clean":"2024-03-14T22:05:15.551031+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T22:05:15.551031+0000","last_undegraded":"2024-03-14T22:05:15.551031+0000","last_fullsized":"2024-03-14T22:05:15.551031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4362,"num_write_kb":43693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'9883","reported_seq":9931,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.440163+0000","last_change":"2024-03-14T21:14:03.039549+0000","last_active":"2024-03-14T22:05:16.440163+0000","last_peered":"2024-03-14T22:05:16.440163+0000","last_clean":"2024-03-14T22:05:16.440163+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T22:05:16.440163+0000","last_undegraded":"2024-03-14T22:05:16.440163+0000","last_fullsized":"2024-03-14T22:05:16.440163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9883,"log_dups_size":0,"ondisk_log_size":9883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:23.510516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":8266,"num_write_kb":116331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'6399","reported_seq":6435,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.600135+0000","last_change":"2024-03-14T21:14:02.660292+0000","last_active":"2024-03-14T22:05:16.600135+0000","last_peered":"2024-03-14T22:05:16.600135+0000","last_clean":"2024-03-14T22:05:16.600135+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T22:05:16.600135+0000","last_undegraded":"2024-03-14T22:05:16.600135+0000","last_fullsized":"2024-03-14T22:05:16.600135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6399,"log_dups_size":0,"ondisk_log_size":6399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6417,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'10919","reported_seq":10954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.435069+0000","last_change":"2024-03-14T21:14:03.040817+0000","last_active":"2024-03-14T22:05:17.435069+0000","last_peered":"2024-03-14T22:05:17.435069+0000","last_clean":"2024-03-14T22:05:17.435069+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T22:05:17.435069+0000","last_undegraded":"2024-03-14T22:05:17.435069+0000","last_fullsized":"2024-03-14T22:05:17.435069+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":120976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'9349","reported_seq":9385,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:01:50.712636+0000","last_change":"2024-03-14T21:14:02.661073+0000","last_active":"2024-03-14T22:01:50.712636+0000","last_peered":"2024-03-14T22:01:50.712636+0000","last_clean":"2024-03-14T22:01:50.712636+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T22:01:50.712636+0000","last_undegraded":"2024-03-14T22:01:50.712636+0000","last_fullsized":"2024-03-14T22:01:50.712636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9349,"log_dups_size":0,"ondisk_log_size":9349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9372,"num_write_kb":71713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'10191","reported_seq":10229,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.350982+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T22:05:17.350982+0000","last_peered":"2024-03-14T22:05:17.350982+0000","last_clean":"2024-03-14T22:05:17.350982+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T22:05:17.350982+0000","last_undegraded":"2024-03-14T22:05:17.350982+0000","last_fullsized":"2024-03-14T22:05:17.350982+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8519,"num_write_kb":106953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'11400","reported_seq":11835,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.412968+0000","last_change":"2024-03-14T21:14:02.663131+0000","last_active":"2024-03-14T22:05:23.412968+0000","last_peered":"2024-03-14T22:05:23.412968+0000","last_clean":"2024-03-14T22:05:23.412968+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T22:05:23.412968+0000","last_undegraded":"2024-03-14T22:05:23.412968+0000","last_fullsized":"2024-03-14T22:05:23.412968+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13689487,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":401,"num_read_kb":1362,"num_write":11406,"num_write_kb":86543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'13457","reported_seq":13498,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.396876+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T22:05:17.396876+0000","last_peered":"2024-03-14T22:05:17.396876+0000","last_clean":"2024-03-14T22:05:17.396876+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T22:05:17.396876+0000","last_undegraded":"2024-03-14T22:05:17.396876+0000","last_fullsized":"2024-03-14T22:05:17.396876+0000","mapping_epoch":19,"log_start":"23'3400","ondisk_log_start":"23'3400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":11847,"num_write_kb":149794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'10672","reported_seq":10705,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.469190+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T22:05:16.469190+0000","last_peered":"2024-03-14T22:05:16.469190+0000","last_clean":"2024-03-14T22:05:16.469190+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T22:05:16.469190+0000","last_undegraded":"2024-03-14T22:05:16.469190+0000","last_fullsized":"2024-03-14T22:05:16.469190+0000","mapping_epoch":19,"log_start":"23'600","ondisk_log_start":"23'600","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":8986,"num_write_kb":111494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6864","reported_seq":6897,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.409531+0000","last_change":"2024-03-14T21:14:02.660393+0000","last_active":"2024-03-14T22:05:23.409531+0000","last_peered":"2024-03-14T22:05:23.409531+0000","last_clean":"2024-03-14T22:05:23.409531+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T22:05:23.409531+0000","last_undegraded":"2024-03-14T22:05:23.409531+0000","last_fullsized":"2024-03-14T22:05:23.409531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6864,"log_dups_size":0,"ondisk_log_size":6864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6885,"num_write_kb":66562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'11241","reported_seq":11278,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.351892+0000","last_change":"2024-03-14T21:14:44.859446+0000","last_active":"2024-03-14T22:05:17.351892+0000","last_peered":"2024-03-14T22:05:17.351892+0000","last_clean":"2024-03-14T22:05:17.351892+0000","last_became_active":"2024-03-14T21:14:44.859128+0000","last_became_peered":"2024-03-14T21:14:44.859128+0000","last_unstale":"2024-03-14T22:05:17.351892+0000","last_undegraded":"2024-03-14T22:05:17.351892+0000","last_fullsized":"2024-03-14T22:05:17.351892+0000","mapping_epoch":22,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:32:11.377675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1780,"num_write":9572,"num_write_kb":116659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15422","reported_seq":16524,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.386679+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T22:05:23.386679+0000","last_peered":"2024-03-14T22:05:23.386679+0000","last_clean":"2024-03-14T22:05:23.386679+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T22:05:23.386679+0000","last_undegraded":"2024-03-14T22:05:23.386679+0000","last_fullsized":"2024-03-14T22:05:23.386679+0000","mapping_epoch":18,"log_start":"23'5400","ondisk_log_start":"23'5400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:48.203059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1069,"num_read_kb":4840,"num_write":15404,"num_write_kb":112501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'10425","reported_seq":10472,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.447186+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T22:05:17.447186+0000","last_peered":"2024-03-14T22:05:17.447186+0000","last_clean":"2024-03-14T22:05:17.447186+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T22:05:17.447186+0000","last_undegraded":"2024-03-14T22:05:17.447186+0000","last_fullsized":"2024-03-14T22:05:17.447186+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2796,"num_write":8857,"num_write_kb":109660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'8706","reported_seq":8744,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.407645+0000","last_change":"2024-03-14T21:14:02.654963+0000","last_active":"2024-03-14T22:05:23.407645+0000","last_peered":"2024-03-14T22:05:23.407645+0000","last_clean":"2024-03-14T22:05:23.407645+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T22:05:23.407645+0000","last_undegraded":"2024-03-14T22:05:23.407645+0000","last_fullsized":"2024-03-14T22:05:23.407645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8706,"log_dups_size":0,"ondisk_log_size":8706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8698,"num_write_kb":67327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'11830","reported_seq":11868,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.429804+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T22:05:17.429804+0000","last_peered":"2024-03-14T22:05:17.429804+0000","last_clean":"2024-03-14T22:05:17.429804+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T22:05:17.429804+0000","last_undegraded":"2024-03-14T22:05:17.429804+0000","last_fullsized":"2024-03-14T22:05:17.429804+0000","mapping_epoch":19,"log_start":"23'1800","ondisk_log_start":"23'1800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":10179,"num_write_kb":107836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6046","reported_seq":6083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.408036+0000","last_change":"2024-03-14T21:14:02.656125+0000","last_active":"2024-03-14T22:05:23.408036+0000","last_peered":"2024-03-14T22:05:23.408036+0000","last_clean":"2024-03-14T22:05:23.408036+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T22:05:23.408036+0000","last_undegraded":"2024-03-14T22:05:23.408036+0000","last_fullsized":"2024-03-14T22:05:23.408036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6048,"num_write_kb":48854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2628,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":345,"num_read_kb":87814,"num_write":552755,"num_write_kb":7123402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624176,"ondisk_log_size":624176,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":256812718,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":3940,"num_read_kb":24158,"num_write":290939,"num_write_kb":2282021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":513675264,"data_stored":513612702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271768,"ondisk_log_size":271768,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575471,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252664,"kb_used_data":112496,"kb_used_omap":1,"kb_used_meta":1140158,"kb_avail":93119176,"statfs":{"total":96636764160,"available":95354036224,"internally_reserved":0,"allocated":115195904,"data_stored":115031492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1167522059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45500000000000002,"15min":0.48699999999999999},"min":{"1min":0.26100000000000001,"5min":0.23000000000000001,"15min":0.19500000000000001},"max":{"1min":0.57499999999999996,"5min":2.782,"15min":4.7999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.63900000000000001,"5min":2.8519999999999999,"15min":3.9239999999999999},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":0.66000000000000003,"5min":1.8520000000000001,"15min":3.903},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51100000000000001,"15min":0.55900000000000005},"min":{"1min":0.30099999999999999,"5min":0.28599999999999998,"15min":0.26400000000000001},"max":{"1min":0.60599999999999998,"5min":6.1440000000000001,"15min":6.6689999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":0.63500000000000001,"5min":3.8210000000000002,"15min":4.8310000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.59199999999999997,"15min":0.54700000000000004},"min":{"1min":0.32000000000000001,"5min":0.29799999999999999,"15min":0.25900000000000001},"max":{"1min":0.59799999999999998,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67200000000000004,"5min":6.1820000000000004,"15min":6.6909999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.5,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.622,"5min":1.9339999999999999,"15min":4.851},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.68700000000000006,"5min":1.226,"15min":6.7220000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.44700000000000001},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.23300000000000001},"max":{"1min":0.69899999999999995,"5min":1.492,"15min":5.2679999999999998},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42999999999999999,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.255,"15min":0.24199999999999999},"max":{"1min":0.71799999999999997,"5min":1.3089999999999999,"15min":3.8740000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.755,"5min":1.321,"15min":6.6459999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.49399999999999999},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69599999999999995,"5min":1.2529999999999999,"15min":4.157},"last":0.442},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.55800000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":1.2509999999999999,"15min":6.6109999999999998},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575470,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101820,"kb_used_data":69876,"kb_used_omap":1,"kb_used_meta":1031934,"kb_avail":93270020,"statfs":{"total":96636764160,"available":95508500480,"internally_reserved":0,"allocated":71553024,"data_stored":71391599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1056700683},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.47299999999999998,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.188},"max":{"1min":2.899,"5min":2.899,"15min":17.885999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.23000000000000001},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.52000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.216},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":7.226},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":1.224,"15min":7.1699999999999999},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.6189999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.26800000000000002},"max":{"1min":3.282,"5min":3.282,"15min":7.2489999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":1.2470000000000001,"15min":7.1470000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.61799999999999999,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.29899999999999999},"max":{"1min":6.1660000000000004,"5min":6.1660000000000004,"15min":6.1660000000000004},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.468,"5min":2.468,"15min":7.3330000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.49199999999999999,"15min":0.45300000000000001},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":6.0229999999999997,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":8.234,"5min":8.234,"15min":8.234},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48799999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.90600000000000003,"5min":1.3340000000000001,"15min":7.5380000000000003},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.54100000000000004,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":7.2039999999999997},"last":0.80800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575470,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1147640,"kb_used_data":69872,"kb_used_omap":1,"kb_used_meta":1077758,"kb_avail":93224200,"statfs":{"total":96636764160,"available":95461580800,"internally_reserved":0,"allocated":71548928,"data_stored":71391017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1103624459},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.55500000000000005},"min":{"1min":0.27300000000000002,"5min":0.221,"15min":0.192},"max":{"1min":0.71199999999999997,"5min":3.2149999999999999,"15min":7.0990000000000002},"last":0.438},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.57499999999999996},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.21099999999999999},"max":{"1min":0.66800000000000004,"5min":5.7359999999999998,"15min":12.281000000000001},"last":0.318}]},{"osd":1,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.68100000000000005,"15min":0.60399999999999998},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.20599999999999999},"max":{"1min":0.70299999999999996,"5min":14.952,"15min":14.952},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49299999999999999,"15min":0.54500000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.222},"max":{"1min":0.63600000000000001,"5min":3.1339999999999999,"15min":8.8539999999999992},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.75800000000000001,"15min":0.63100000000000001},"min":{"1min":0.28699999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":0.70599999999999996,"5min":18.623000000000001,"15min":18.623000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.96399999999999997,"15min":0.70699999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.67000000000000004,"5min":30.004999999999999,"15min":30.004999999999999},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.25},"max":{"1min":0.66200000000000003,"5min":3.6600000000000001,"15min":9.2309999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.57699999999999996},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":0.82699999999999996,"15min":7.3780000000000001},"last":0.498}]},{"osd":4,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.59899999999999998,"15min":0.53700000000000003},"min":{"1min":0.22700000000000001,"5min":0.219,"15min":0.20999999999999999},"max":{"1min":0.58399999999999996,"5min":12.414,"15min":12.414},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.496,"15min":0.57099999999999995},"min":{"1min":0.308,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60799999999999998,"5min":3.1970000000000001,"15min":8.9220000000000006},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.44600000000000001,"15min":0.46600000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.22},"max":{"1min":0.55600000000000005,"5min":1.905,"15min":7.4000000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.61599999999999999,"15min":0.58299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.24299999999999999},"max":{"1min":0.65700000000000003,"5min":10.278,"15min":10.278},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.52300000000000002,"15min":0.55100000000000005},"min":{"1min":0.27500000000000002,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":0.64200000000000002,"5min":3.9820000000000002,"15min":7.1619999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.28399999999999997},"max":{"1min":0.623,"5min":3.7789999999999999,"15min":7.3310000000000004},"last":0.57199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":961852,"kb_used_data":42292,"kb_used_omap":1,"kb_used_meta":919550,"kb_avail":93409988,"statfs":{"total":96636764160,"available":95651827712,"internally_reserved":0,"allocated":43307008,"data_stored":43137500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":941619467},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.495,"15min":0.55600000000000005},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":0.73799999999999999,"5min":2.23,"15min":15.013},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.26000000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.72599999999999998,"5min":2.1139999999999999,"15min":5.6879999999999997},"last":0.628}]},{"osd":1,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":1.806,"15min":4.6479999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.52000000000000002},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":0.64900000000000002,"5min":0.77800000000000002,"15min":4.343},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.503},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70699999999999996,"5min":0.97099999999999997,"15min":5.5540000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57499999999999996,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.26000000000000001},"max":{"1min":0.75700000000000001,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.5}]},{"osd":3,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.22},"max":{"1min":0.67700000000000005,"5min":2.1320000000000001,"15min":9.9800000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.33000000000000002,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":0.77200000000000002,"5min":1.8919999999999999,"15min":5.899},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41499999999999998,"15min":0.40999999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.20599999999999999},"max":{"1min":0.61099999999999999,"5min":0.71599999999999997,"15min":3.0470000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.41899999999999998,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.61699999999999999,"5min":0.69999999999999996,"15min":4.2460000000000004},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44500000000000001,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.76100000000000001,"5min":0.90900000000000003,"15min":3.8650000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45700000000000002,"15min":0.45900000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":1.5680000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.23400000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":8.7639999999999993},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.27400000000000002},"max":{"1min":0.69399999999999995,"5min":0.94799999999999995,"15min":3.7850000000000001},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1131324,"kb_used_data":57588,"kb_used_omap":1,"kb_used_meta":1073726,"kb_avail":93240516,"statfs":{"total":96636764160,"available":95478288384,"internally_reserved":0,"allocated":58970112,"data_stored":58808687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1099495691},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.57599999999999996,"15min":0.53000000000000003},"min":{"1min":0.23000000000000001,"5min":0.20499999999999999,"15min":0.17000000000000001},"max":{"1min":0.52300000000000002,"5min":7.7270000000000003,"15min":14.695},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.186},"max":{"1min":0.51900000000000002,"5min":7.9770000000000003,"15min":8.9649999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.378,"15min":0.45500000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":0.71799999999999997,"5min":0.93500000000000005,"15min":5.5289999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.53700000000000003,"15min":0.60699999999999998},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.20100000000000001},"max":{"1min":0.83899999999999997,"5min":6.1740000000000004,"15min":14.898},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.61599999999999999,"15min":0.57099999999999995},"min":{"1min":0.251,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.83099999999999996,"5min":9.4079999999999995,"15min":9.5150000000000006},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.55900000000000005},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":2.9430000000000001,"15min":15.263},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.71899999999999997,"15min":0.74399999999999999},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":0.77700000000000002,"5min":9.6080000000000005,"15min":14.071999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.622,"15min":0.69499999999999995},"min":{"1min":0.40200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":3.5710000000000002,"15min":14.109},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.70899999999999996,"15min":0.68200000000000005},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":11.443,"15min":11.443},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.74199999999999999,"15min":0.77000000000000002},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.78000000000000003,"5min":7.6399999999999997,"15min":14.550000000000001},"last":0.85399999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.82999999999999996,"15min":0.76500000000000001},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.29299999999999998},"max":{"1min":0.78400000000000003,"5min":9.7200000000000006,"15min":9.7200000000000006},"last":0.873},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.78200000000000003,"15min":0.69899999999999995},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":15.339,"15min":15.339},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.81599999999999995,"15min":0.77700000000000002},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":7.7030000000000003,"15min":14.726000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.83399999999999996,"15min":0.76300000000000001},"min":{"1min":0.433,"5min":0.379,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":9.5079999999999991,"15min":9.5079999999999991},"last":0.91400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575471,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649700,"kb_used_data":45148,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":93722140,"statfs":{"total":96636764160,"available":95971471360,"internally_reserved":0,"allocated":46231552,"data_stored":46077232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":619051275},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.61299999999999999,"15min":0.45800000000000002},"min":{"1min":0.24399999999999999,"5min":0.184,"15min":0.182},"max":{"1min":2.347,"5min":10.869,"15min":10.869},"last":0.311},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.67800000000000005,"15min":0.55200000000000005},"min":{"1min":0.23799999999999999,"5min":0.20100000000000001,"15min":0.189},"max":{"1min":1.107,"5min":9.6649999999999991,"15min":12.317},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.53100000000000003,"15min":0.45600000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":2.0049999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.20100000000000001},"max":{"1min":0.96799999999999997,"5min":7.8090000000000002,"15min":9.5429999999999993},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.66700000000000004,"15min":0.61899999999999999},"min":{"1min":0.26500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56499999999999995,"5min":13.130000000000001,"15min":27.355},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52800000000000002,"15min":0.57099999999999995},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.26000000000000001},"max":{"1min":1.929,"5min":3.4390000000000001,"15min":12.451000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83099999999999996,"15min":0.71999999999999997},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.28899999999999998},"max":{"1min":7.1310000000000002,"5min":8.0440000000000005,"15min":12.478999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.79800000000000004,"15min":0.73699999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.30599999999999999},"max":{"1min":0.86899999999999999,"5min":7.9409999999999998,"15min":12.391999999999999},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.0529999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":0.97299999999999998,"5min":20.84,"15min":20.84},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69199999999999995,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.0329999999999999,"5min":4.7240000000000002,"15min":11.762},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.69699999999999995,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.79800000000000004,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.78000000000000003,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.9590000000000001,"5min":13.177,"15min":13.177},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.79400000000000004,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.35099999999999998,"15min":0.32600000000000001},"max":{"1min":1.8420000000000001,"5min":6.2309999999999999,"15min":11.865},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1031156,"kb_used_data":57452,"kb_used_omap":1,"kb_used_meta":973694,"kb_avail":93340684,"statfs":{"total":96636764160,"available":95580860416,"internally_reserved":0,"allocated":58830848,"data_stored":58655239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":997062923},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49399999999999999,"15min":0.439},"min":{"1min":0.19400000000000001,"5min":0.186,"15min":0.184},"max":{"1min":0.89000000000000001,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.219,"5min":0.219,"15min":0.19700000000000001},"max":{"1min":2.145,"5min":6.5339999999999998,"15min":9.2249999999999996},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54800000000000004,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":2.161,"5min":13.553000000000001,"15min":13.553000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":2.0910000000000002,"5min":6.6050000000000004,"15min":7.1180000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.70699999999999996,"15min":0.64000000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.22900000000000001},"max":{"1min":2.1190000000000002,"5min":10.236000000000001,"15min":24.879000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.53400000000000003},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.22900000000000001},"max":{"1min":1.4670000000000001,"5min":2.2290000000000001,"15min":20.995000000000001},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":7.1609999999999996,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":8.4420000000000002,"5min":8.4420000000000002,"15min":8.4420000000000002},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9059999999999999,"5min":3.5800000000000001,"15min":8.6419999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.79100000000000004,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.96699999999999997,"5min":16.943999999999999,"15min":16.943999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.0270000000000001,"5min":3.5409999999999999,"15min":7.468},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1,"5min":3.4950000000000001,"15min":5.5030000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":2.5169999999999999,"5min":8.593,"15min":9.3209999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.40200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.496,"5min":6.7859999999999996,"15min":10.346},"last":0.80400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":871104,"kb_used_data":49976,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":93500736,"statfs":{"total":96636764160,"available":95744753664,"internally_reserved":0,"allocated":51175424,"data_stored":51009794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":840825099},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.45500000000000002},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.17499999999999999},"max":{"1min":2.8149999999999999,"5min":12.734,"15min":12.734},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52000000000000002,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":3.976,"5min":9.3620000000000001,"15min":9.3620000000000001},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.843,"5min":15.375,"15min":15.375},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.48099999999999998},"min":{"1min":0.28399999999999997,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.008,"5min":9.2959999999999994,"15min":9.2959999999999994},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.20699999999999999},"max":{"1min":2.7040000000000002,"5min":8.7230000000000008,"15min":8.7230000000000008},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":2.9169999999999998,"5min":10.928000000000001,"15min":12.864000000000001},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0350000000000001,"5min":9.2319999999999993,"15min":9.2319999999999993},"last":0.435},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":3.7930000000000001,"15min":7.5499999999999998},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.61899999999999999},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":3.0790000000000002,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.68700000000000006},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"max":{"1min":2.8660000000000001,"5min":10.135999999999999,"15min":11.068},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.69699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.8959999999999999,"5min":12.77,"15min":12.77},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":4.0579999999999998,"5min":9.391,"15min":9.391},"last":0.375}]},{"osd":7,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73299999999999998,"15min":0.67900000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.0830000000000002,"5min":9.3290000000000006,"15min":9.3290000000000006},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.71499999999999997,"15min":0.65400000000000003},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.33300000000000002},"max":{"1min":0.78900000000000003,"5min":7.8739999999999997,"15min":7.8739999999999997},"last":0.47399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58568704,"data_stored":58559811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45989888,"data_stored":45989473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43065344,"data_stored":43049741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":71307264,"data_stored":71303258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":71311360,"data_stored":71303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114360320,"data_stored":114353365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T22:05:27.048 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-14T22:05:27.049 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T22:05:27.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.192+0000 7fdcf432e640 1 Processor -- start 2024-03-14T22:05:27.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.192+0000 7fdcf432e640 1 -- start start 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf432e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcec06b620 0x7fdcec0bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf432e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec06a6e0 con 0x7fdcec0baef0 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf432e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec0bc2b0 con 0x7fdcec0bc940 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf432e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcec06ad10 con 0x7fdcec06b620 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf18c9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcec06b620 0x7fdcec0bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf18c9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcec06b620 0x7fdcec0bf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50140/0 (socket says 172.21.15.49:50140) 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf20ca640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdcec0baef0 0x7fdcec1480b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43572/0 (socket says 172.21.15.49:43572) 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.193+0000 7fdcf18c9640 1 -- 172.21.15.49:0/352297414 learned_addr learned my addr 172.21.15.49:0/352297414 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.194+0000 7fdcf18c9640 1 -- 172.21.15.49:0/352297414 >> v1:172.21.15.97:6789/0 conn(0x7fdcec0bc940 legacy=0x7fdcec155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.194+0000 7fdcf18c9640 1 -- 172.21.15.49:0/352297414 >> v1:172.21.15.49:6789/0 conn(0x7fdcec0baef0 legacy=0x7fdcec1480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.194+0000 7fdcf18c9640 1 -- 172.21.15.49:0/352297414 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec06bab0 con 0x7fdcec06b620 2024-03-14T22:05:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.194+0000 7fdcf18c9640 1 --2- 172.21.15.49:0/352297414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec06b620 0x7fdcec0bf7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009870 tx=0x7fdce802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaebb2280471945 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf10c8640 1 -- 172.21.15.49:0/352297414 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8030860 con 0x7fdcec06b620 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf10c8640 1 -- 172.21.15.49:0/352297414 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce80309c0 con 0x7fdcec06b620 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf10c8640 1 -- 172.21.15.49:0/352297414 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8030d50 con 0x7fdcec06b620 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf432e640 1 -- 172.21.15.49:0/352297414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec06b620 msgr2=0x7fdcec0bf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf432e640 1 --2- 172.21.15.49:0/352297414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec06b620 0x7fdcec0bf7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009870 tx=0x7fdce802b9a0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf432e640 1 -- 172.21.15.49:0/352297414 shutdown_connections 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf432e640 1 --2- 172.21.15.49:0/352297414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec06b620 0x7fdcec0bf7f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf432e640 1 -- 172.21.15.49:0/352297414 >> 172.21.15.49:0/352297414 conn(0x7fdcec0bcd80 msgr2=0x7fdcec0bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf432e640 1 -- 172.21.15.49:0/352297414 shutdown_connections 2024-03-14T22:05:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.195+0000 7fdcf432e640 1 -- 172.21.15.49:0/352297414 wait complete. 2024-03-14T22:05:27.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.196+0000 7fdcf432e640 1 Processor -- start 2024-03-14T22:05:27.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.196+0000 7fdcf432e640 1 -- start start 2024-03-14T22:05:27.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf432e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcec0baef0 0x7fdcec0a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf432e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcec0bc940 0x7fdcec09d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf432e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec0b65c0 0x7fdcec099630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf432e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0a2c70 con 0x7fdcec0b65c0 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf432e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec09e4b0 con 0x7fdcec0baef0 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf432e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec09e950 con 0x7fdcec0bc940 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec0b65c0 0x7fdcec099630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec0b65c0 0x7fdcec099630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50146/0 (socket says 172.21.15.49:50146) 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 -- 172.21.15.49:0/983319435 learned_addr learned my addr 172.21.15.49:0/983319435 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf20ca640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcec0baef0 0x7fdcec0a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf18c9640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcec0bc940 0x7fdcec09d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 -- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcec0bc940 msgr2=0x7fdcec09d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcec0bc940 0x7fdcec09d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 -- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcec0baef0 msgr2=0x7fdcec0a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcec0baef0 0x7fdcec0a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.197+0000 7fdcf28cb640 1 -- 172.21.15.49:0/983319435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80094d0 con 0x7fdcec0b65c0 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.198+0000 7fdcf28cb640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec0b65c0 0x7fdcec099630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c6b0 tx=0x7fdce400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.198+0000 7fdce2ffd640 1 -- 172.21.15.49:0/983319435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003b70 con 0x7fdcec0b65c0 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.198+0000 7fdce2ffd640 1 -- 172.21.15.49:0/983319435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce4003cf0 con 0x7fdcec0b65c0 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.198+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec148940 con 0x7fdcec0b65c0 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.198+0000 7fdce2ffd640 1 -- 172.21.15.49:0/983319435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003ea0 con 0x7fdcec0b65c0 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.198+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec09f180 con 0x7fdcec0b65c0 2024-03-14T22:05:27.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.199+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec066bb0 con 0x7fdcec0b65c0 2024-03-14T22:05:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.199+0000 7fdce2ffd640 1 -- 172.21.15.49:0/983319435 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fdce4005310 con 0x7fdcec0b65c0 2024-03-14T22:05:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.199+0000 7fdce2ffd640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcd006a7d0 0x7fdcd006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.200+0000 7fdce2ffd640 1 -- 172.21.15.49:0/983319435 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdce4088b20 con 0x7fdcec0b65c0 2024-03-14T22:05:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.200+0000 7fdcf20ca640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcd006a7d0 0x7fdcd006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.200+0000 7fdcf20ca640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcd006a7d0 0x7fdcd006cc90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc000f60 tx=0x7fdcdc007c80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:27.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.204+0000 7fdce2ffd640 1 -- 172.21.15.49:0/983319435 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdce4053f50 con 0x7fdcec0b65c0 2024-03-14T22:05:27.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.380+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdcec1487c0 con 0x7fdcd006a7d0 2024-03-14T22:05:27.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.388+0000 7fdce2ffd640 1 -- 172.21.15.49:0/983319435 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292078 (secure 0 0 0) 0x7fdcdc005a50 con 0x7fdcd006a7d0 2024-03-14T22:05:27.390 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:05:27.391 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcd006a7d0 msgr2=0x7fdcd006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcd006a7d0 0x7fdcd006cc90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc000f60 tx=0x7fdcdc007c80 comp rx=0 tx=0).stop 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec0b65c0 msgr2=0x7fdcec099630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec0b65c0 0x7fdcec099630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c6b0 tx=0x7fdce400cb70 comp rx=0 tx=0).stop 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 shutdown_connections 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fdcd006a7d0 0x7fdcd006cc90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcec0b65c0 0x7fdcec099630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcec0bc940 0x7fdcec09d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 --2- 172.21.15.49:0/983319435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcec0baef0 0x7fdcec0a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 >> 172.21.15.49:0/983319435 conn(0x7fdcec0bcd80 msgr2=0x7fdcec148ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 shutdown_connections 2024-03-14T22:05:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.391+0000 7fdcf432e640 1 -- 172.21.15.49:0/983319435 wait complete. 2024-03-14T22:05:27.401 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":1564,"stamp":"2024-03-14T22:05:25.588581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257405714,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":4401,"num_read_kb":112072,"num_write":843853,"num_write_kb":9407839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896061,"ondisk_log_size":896061,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8147260,"kb_used_data":504700,"kb_used_omap":13,"kb_used_meta":7642482,"kb_avail":746827460,"statfs":{"total":773094113280,"available":764751319040,"internally_reserved":0,"allocated":516812800,"data_stored":515502560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":7825901656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11540782,"num_objects":-2092,"num_object_clones":0,"num_object_copies":-4184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2092,"num_whiteouts":0,"num_read":14,"num_read_kb":226,"num_write":2321,"num_write_kb":8782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006146"},"pg_stats":[{"pgid":"3.3f","version":"23'10328","reported_seq":10370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.391677+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T22:05:17.391677+0000","last_peered":"2024-03-14T22:05:17.391677+0000","last_clean":"2024-03-14T22:05:17.391677+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T22:05:17.391677+0000","last_undegraded":"2024-03-14T22:05:17.391677+0000","last_fullsized":"2024-03-14T22:05:17.391677+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:57:09.325211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8785,"num_write_kb":106962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'9093","reported_seq":9142,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.446982+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T22:05:17.446982+0000","last_peered":"2024-03-14T22:05:17.446982+0000","last_clean":"2024-03-14T22:05:17.446982+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T22:05:17.446982+0000","last_undegraded":"2024-03-14T22:05:17.446982+0000","last_fullsized":"2024-03-14T22:05:17.446982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9093,"log_dups_size":0,"ondisk_log_size":9093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":7584,"num_write_kb":103945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'11562","reported_seq":11603,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.356641+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T22:05:23.356641+0000","last_peered":"2024-03-14T22:05:23.356641+0000","last_clean":"2024-03-14T22:05:23.356641+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T22:05:23.356641+0000","last_undegraded":"2024-03-14T22:05:23.356641+0000","last_fullsized":"2024-03-14T22:05:23.356641+0000","mapping_epoch":19,"log_start":"23'1500","ondisk_log_start":"23'1500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:08:55.492956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1274,"num_write":9914,"num_write_kb":127294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'10095","reported_seq":10141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.609893+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T22:05:16.609893+0000","last_peered":"2024-03-14T22:05:16.609893+0000","last_clean":"2024-03-14T22:05:16.609893+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T22:05:16.609893+0000","last_undegraded":"2024-03-14T22:05:16.609893+0000","last_fullsized":"2024-03-14T22:05:16.609893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8407,"num_write_kb":110879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'9604","reported_seq":9638,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.547040+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T22:05:16.547040+0000","last_peered":"2024-03-14T22:05:16.547040+0000","last_clean":"2024-03-14T22:05:16.547040+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T22:05:16.547040+0000","last_undegraded":"2024-03-14T22:05:16.547040+0000","last_fullsized":"2024-03-14T22:05:16.547040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9604,"log_dups_size":0,"ondisk_log_size":9604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8008,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'9639","reported_seq":9679,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.400588+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T22:05:17.400588+0000","last_peered":"2024-03-14T22:05:17.400588+0000","last_clean":"2024-03-14T22:05:17.400588+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T22:05:17.400588+0000","last_undegraded":"2024-03-14T22:05:17.400588+0000","last_fullsized":"2024-03-14T22:05:17.400588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":8054,"num_write_kb":114907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10032","reported_seq":10068,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431473+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T22:05:17.431473+0000","last_peered":"2024-03-14T22:05:17.431473+0000","last_clean":"2024-03-14T22:05:17.431473+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T22:05:17.431473+0000","last_undegraded":"2024-03-14T22:05:17.431473+0000","last_fullsized":"2024-03-14T22:05:17.431473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":8423,"num_write_kb":104789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'11190","reported_seq":11230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.614200+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T22:05:16.614200+0000","last_peered":"2024-03-14T22:05:16.614200+0000","last_clean":"2024-03-14T22:05:16.614200+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T22:05:16.614200+0000","last_undegraded":"2024-03-14T22:05:16.614200+0000","last_fullsized":"2024-03-14T22:05:16.614200+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":9605,"num_write_kb":125652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'9011","reported_seq":9048,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.436028+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T22:05:17.436028+0000","last_peered":"2024-03-14T22:05:17.436028+0000","last_clean":"2024-03-14T22:05:17.436028+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T22:05:17.436028+0000","last_undegraded":"2024-03-14T22:05:17.436028+0000","last_fullsized":"2024-03-14T22:05:17.436028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9011,"log_dups_size":0,"ondisk_log_size":9011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7381,"num_write_kb":104568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'9532","reported_seq":9568,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.411246+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T22:05:17.411246+0000","last_peered":"2024-03-14T22:05:17.411246+0000","last_clean":"2024-03-14T22:05:17.411246+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T22:05:17.411246+0000","last_undegraded":"2024-03-14T22:05:17.411246+0000","last_fullsized":"2024-03-14T22:05:17.411246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7843,"num_write_kb":113016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'9221","reported_seq":9262,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.399007+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T22:05:17.399007+0000","last_peered":"2024-03-14T22:05:17.399007+0000","last_clean":"2024-03-14T22:05:17.399007+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T22:05:17.399007+0000","last_undegraded":"2024-03-14T22:05:17.399007+0000","last_fullsized":"2024-03-14T22:05:17.399007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7677,"num_write_kb":104837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'10300","reported_seq":10335,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431722+0000","last_change":"2024-03-14T21:14:03.034043+0000","last_active":"2024-03-14T22:05:17.431722+0000","last_peered":"2024-03-14T22:05:17.431722+0000","last_clean":"2024-03-14T22:05:17.431722+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T22:05:17.431722+0000","last_undegraded":"2024-03-14T22:05:17.431722+0000","last_fullsized":"2024-03-14T22:05:17.431722+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:33.787920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8730,"num_write_kb":105734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'8860","reported_seq":8898,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.540388+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T22:05:16.540388+0000","last_peered":"2024-03-14T22:05:16.540388+0000","last_clean":"2024-03-14T22:05:16.540388+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T22:05:16.540388+0000","last_undegraded":"2024-03-14T22:05:16.540388+0000","last_fullsized":"2024-03-14T22:05:16.540388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8860,"log_dups_size":0,"ondisk_log_size":8860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":7264,"num_write_kb":95419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'10404","reported_seq":10443,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.392084+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T22:05:17.392084+0000","last_peered":"2024-03-14T22:05:17.392084+0000","last_clean":"2024-03-14T22:05:17.392084+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T22:05:17.392084+0000","last_undegraded":"2024-03-14T22:05:17.392084+0000","last_fullsized":"2024-03-14T22:05:17.392084+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8793,"num_write_kb":111405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'9697","reported_seq":9734,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.610064+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T22:05:16.610064+0000","last_peered":"2024-03-14T22:05:16.610064+0000","last_clean":"2024-03-14T22:05:16.610064+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T22:05:16.610064+0000","last_undegraded":"2024-03-14T22:05:16.610064+0000","last_fullsized":"2024-03-14T22:05:16.610064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9697,"log_dups_size":0,"ondisk_log_size":9697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8012,"num_write_kb":95820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'9656","reported_seq":9704,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.455534+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T22:05:17.455534+0000","last_peered":"2024-03-14T22:05:17.455534+0000","last_clean":"2024-03-14T22:05:17.455534+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T22:05:17.455534+0000","last_undegraded":"2024-03-14T22:05:17.455534+0000","last_fullsized":"2024-03-14T22:05:17.455534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9656,"log_dups_size":0,"ondisk_log_size":9656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3572,"num_write":8076,"num_write_kb":107857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8658","reported_seq":8695,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.457733+0000","last_change":"2024-03-14T21:14:44.859268+0000","last_active":"2024-03-14T22:05:17.457733+0000","last_peered":"2024-03-14T22:05:17.457733+0000","last_clean":"2024-03-14T22:05:17.457733+0000","last_became_active":"2024-03-14T21:14:44.858906+0000","last_became_peered":"2024-03-14T21:14:44.858906+0000","last_unstale":"2024-03-14T22:05:17.457733+0000","last_undegraded":"2024-03-14T22:05:17.457733+0000","last_fullsized":"2024-03-14T22:05:17.457733+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8658,"log_dups_size":0,"ondisk_log_size":8658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:56:07.747779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1528,"num_write":7020,"num_write_kb":102847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10194","reported_seq":10230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.574269+0000","last_change":"2024-03-14T21:14:03.029823+0000","last_active":"2024-03-14T22:05:16.574269+0000","last_peered":"2024-03-14T22:05:16.574269+0000","last_clean":"2024-03-14T22:05:16.574269+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T22:05:16.574269+0000","last_undegraded":"2024-03-14T22:05:16.574269+0000","last_fullsized":"2024-03-14T22:05:16.574269+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:47.131679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8552,"num_write_kb":105487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'10515","reported_seq":10549,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.365482+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T22:05:23.365482+0000","last_peered":"2024-03-14T22:05:23.365482+0000","last_clean":"2024-03-14T22:05:23.365482+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T22:05:23.365482+0000","last_undegraded":"2024-03-14T22:05:23.365482+0000","last_fullsized":"2024-03-14T22:05:23.365482+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":118219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'10892","reported_seq":10930,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.430759+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T22:05:17.430759+0000","last_peered":"2024-03-14T22:05:17.430759+0000","last_clean":"2024-03-14T22:05:17.430759+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T22:05:17.430759+0000","last_undegraded":"2024-03-14T22:05:17.430759+0000","last_fullsized":"2024-03-14T22:05:17.430759+0000","mapping_epoch":19,"log_start":"23'800","ondisk_log_start":"23'800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":9315,"num_write_kb":117297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'10951","reported_seq":10997,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.442792+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T22:05:17.442792+0000","last_peered":"2024-03-14T22:05:17.442792+0000","last_clean":"2024-03-14T22:05:17.442792+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T22:05:17.442792+0000","last_undegraded":"2024-03-14T22:05:17.442792+0000","last_fullsized":"2024-03-14T22:05:17.442792+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3050,"num_write":9338,"num_write_kb":109552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10716","reported_seq":10752,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.349639+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T22:05:23.349639+0000","last_peered":"2024-03-14T22:05:23.349639+0000","last_clean":"2024-03-14T22:05:23.349639+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T22:05:23.349639+0000","last_undegraded":"2024-03-14T22:05:23.349639+0000","last_fullsized":"2024-03-14T22:05:23.349639+0000","mapping_epoch":19,"log_start":"23'700","ondisk_log_start":"23'700","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":9066,"num_write_kb":118743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'10557","reported_seq":10593,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.383547+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T22:05:17.383547+0000","last_peered":"2024-03-14T22:05:17.383547+0000","last_clean":"2024-03-14T22:05:17.383547+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T22:05:17.383547+0000","last_undegraded":"2024-03-14T22:05:17.383547+0000","last_fullsized":"2024-03-14T22:05:17.383547+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8950,"num_write_kb":109690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'11172","reported_seq":11213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.420018+0000","last_change":"2024-03-14T21:14:03.038694+0000","last_active":"2024-03-14T22:05:17.420018+0000","last_peered":"2024-03-14T22:05:17.420018+0000","last_clean":"2024-03-14T22:05:17.420018+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T22:05:17.420018+0000","last_undegraded":"2024-03-14T22:05:17.420018+0000","last_fullsized":"2024-03-14T22:05:17.420018+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1530,"num_write":9571,"num_write_kb":127549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'10188","reported_seq":10225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.442741+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T22:05:17.442741+0000","last_peered":"2024-03-14T22:05:17.442741+0000","last_clean":"2024-03-14T22:05:17.442741+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T22:05:17.442741+0000","last_undegraded":"2024-03-14T22:05:17.442741+0000","last_fullsized":"2024-03-14T22:05:17.442741+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":8582,"num_write_kb":110246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'12329","reported_seq":12370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.390032+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T22:05:17.390032+0000","last_peered":"2024-03-14T22:05:17.390032+0000","last_clean":"2024-03-14T22:05:17.390032+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T22:05:17.390032+0000","last_undegraded":"2024-03-14T22:05:17.390032+0000","last_fullsized":"2024-03-14T22:05:17.390032+0000","mapping_epoch":19,"log_start":"23'2300","ondisk_log_start":"23'2300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10697,"num_write_kb":121303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'9238","reported_seq":9281,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.393718+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T22:05:17.393718+0000","last_peered":"2024-03-14T22:05:17.393718+0000","last_clean":"2024-03-14T22:05:17.393718+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T22:05:17.393718+0000","last_undegraded":"2024-03-14T22:05:17.393718+0000","last_fullsized":"2024-03-14T22:05:17.393718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9238,"log_dups_size":0,"ondisk_log_size":9238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2628,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7599,"num_write_kb":100407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'10304","reported_seq":10339,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.430400+0000","last_change":"2024-03-14T21:14:03.040690+0000","last_active":"2024-03-14T22:05:17.430400+0000","last_peered":"2024-03-14T22:05:17.430400+0000","last_clean":"2024-03-14T22:05:17.430400+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T22:05:17.430400+0000","last_undegraded":"2024-03-14T22:05:17.430400+0000","last_fullsized":"2024-03-14T22:05:17.430400+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":108137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'11950","reported_seq":11998,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.459036+0000","last_change":"2024-03-14T21:14:44.746590+0000","last_active":"2024-03-14T22:05:17.459036+0000","last_peered":"2024-03-14T22:05:17.459036+0000","last_clean":"2024-03-14T22:05:17.459036+0000","last_became_active":"2024-03-14T21:14:44.746271+0000","last_became_peered":"2024-03-14T21:14:44.746271+0000","last_unstale":"2024-03-14T22:05:17.459036+0000","last_undegraded":"2024-03-14T22:05:17.459036+0000","last_fullsized":"2024-03-14T22:05:17.459036+0000","mapping_epoch":22,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:01:46.971606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":10306,"num_write_kb":120667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'11949","reported_seq":11991,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431315+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T22:05:17.431315+0000","last_peered":"2024-03-14T22:05:17.431315+0000","last_clean":"2024-03-14T22:05:17.431315+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T22:05:17.431315+0000","last_undegraded":"2024-03-14T22:05:17.431315+0000","last_fullsized":"2024-03-14T22:05:17.431315+0000","mapping_epoch":19,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10416,"num_write_kb":126678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'11288","reported_seq":11335,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.451854+0000","last_change":"2024-03-14T21:14:03.029847+0000","last_active":"2024-03-14T22:05:17.451854+0000","last_peered":"2024-03-14T22:05:17.451854+0000","last_clean":"2024-03-14T22:05:17.451854+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T22:05:17.451854+0000","last_undegraded":"2024-03-14T22:05:17.451854+0000","last_fullsized":"2024-03-14T22:05:17.451854+0000","mapping_epoch":19,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2806,"num_write":9622,"num_write_kb":118224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'8631","reported_seq":8668,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.386380+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T22:05:17.386380+0000","last_peered":"2024-03-14T22:05:17.386380+0000","last_clean":"2024-03-14T22:05:17.386380+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T22:05:17.386380+0000","last_undegraded":"2024-03-14T22:05:17.386380+0000","last_fullsized":"2024-03-14T22:05:17.386380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":7051,"num_write_kb":105885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'9047","reported_seq":9083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431703+0000","last_change":"2024-03-14T21:14:03.037915+0000","last_active":"2024-03-14T22:05:17.431703+0000","last_peered":"2024-03-14T22:05:17.431703+0000","last_clean":"2024-03-14T22:05:17.431703+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T22:05:17.431703+0000","last_undegraded":"2024-03-14T22:05:17.431703+0000","last_fullsized":"2024-03-14T22:05:17.431703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":7364,"num_write_kb":104642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'10861","reported_seq":10894,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422940+0000","last_change":"2024-03-14T21:14:02.655762+0000","last_active":"2024-03-14T22:05:23.422940+0000","last_peered":"2024-03-14T22:05:23.422940+0000","last_clean":"2024-03-14T22:05:23.422940+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T22:05:23.422940+0000","last_undegraded":"2024-03-14T22:05:23.422940+0000","last_fullsized":"2024-03-14T22:05:23.422940+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10855,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'9532","reported_seq":9571,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.556727+0000","last_change":"2024-03-14T21:14:03.033091+0000","last_active":"2024-03-14T22:05:16.556727+0000","last_peered":"2024-03-14T22:05:16.556727+0000","last_clean":"2024-03-14T22:05:16.556727+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T22:05:16.556727+0000","last_undegraded":"2024-03-14T22:05:16.556727+0000","last_fullsized":"2024-03-14T22:05:16.556727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":7923,"num_write_kb":107722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10915","reported_seq":10948,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.394449+0000","last_change":"2024-03-14T21:14:02.662899+0000","last_active":"2024-03-14T22:05:23.394449+0000","last_peered":"2024-03-14T22:05:23.394449+0000","last_clean":"2024-03-14T22:05:23.394449+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T22:05:23.394449+0000","last_undegraded":"2024-03-14T22:05:23.394449+0000","last_fullsized":"2024-03-14T22:05:23.394449+0000","mapping_epoch":18,"log_start":"23'900","ondisk_log_start":"23'900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":10937,"num_write_kb":77181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'7036","reported_seq":7070,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422556+0000","last_change":"2024-03-14T21:14:02.660574+0000","last_active":"2024-03-14T22:05:23.422556+0000","last_peered":"2024-03-14T22:05:23.422556+0000","last_clean":"2024-03-14T22:05:23.422556+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T22:05:23.422556+0000","last_undegraded":"2024-03-14T22:05:23.422556+0000","last_fullsized":"2024-03-14T22:05:23.422556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":7036,"log_dups_size":0,"ondisk_log_size":7036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7046,"num_write_kb":58697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'11484","reported_seq":11518,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.365328+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T22:05:23.365328+0000","last_peered":"2024-03-14T22:05:23.365328+0000","last_clean":"2024-03-14T22:05:23.365328+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T22:05:23.365328+0000","last_undegraded":"2024-03-14T22:05:23.365328+0000","last_fullsized":"2024-03-14T22:05:23.365328+0000","mapping_epoch":19,"log_start":"23'1400","ondisk_log_start":"23'1400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":9890,"num_write_kb":106254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'11076","reported_seq":11110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.352182+0000","last_change":"2024-03-14T21:14:02.661303+0000","last_active":"2024-03-14T22:05:17.352182+0000","last_peered":"2024-03-14T22:05:17.352182+0000","last_clean":"2024-03-14T22:05:17.352182+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T22:05:17.352182+0000","last_undegraded":"2024-03-14T22:05:17.352182+0000","last_fullsized":"2024-03-14T22:05:17.352182+0000","mapping_epoch":18,"log_start":"23'1000","ondisk_log_start":"23'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11078,"num_write_kb":81064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'10471","reported_seq":10515,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.565713+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T22:05:16.565713+0000","last_peered":"2024-03-14T22:05:16.565713+0000","last_clean":"2024-03-14T22:05:16.565713+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T22:05:16.565713+0000","last_undegraded":"2024-03-14T22:05:16.565713+0000","last_fullsized":"2024-03-14T22:05:16.565713+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2800,"num_write":8909,"num_write_kb":114509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8286","reported_seq":8925,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.553161+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T22:05:15.553161+0000","last_peered":"2024-03-14T22:05:15.553161+0000","last_clean":"2024-03-14T22:05:15.553161+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T22:05:15.553161+0000","last_undegraded":"2024-03-14T22:05:15.553161+0000","last_fullsized":"2024-03-14T22:05:15.553161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8286,"log_dups_size":0,"ondisk_log_size":8286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":607,"num_read_kb":4494,"num_write":8292,"num_write_kb":66794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10197","reported_seq":10235,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.586675+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T22:05:16.586675+0000","last_peered":"2024-03-14T22:05:16.586675+0000","last_clean":"2024-03-14T22:05:16.586675+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T22:05:16.586675+0000","last_undegraded":"2024-03-14T22:05:16.586675+0000","last_fullsized":"2024-03-14T22:05:16.586675+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8609,"num_write_kb":104201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'10690","reported_seq":10724,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.418804+0000","last_change":"2024-03-14T21:14:02.660384+0000","last_active":"2024-03-14T22:05:23.418804+0000","last_peered":"2024-03-14T22:05:23.418804+0000","last_clean":"2024-03-14T22:05:23.418804+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T22:05:23.418804+0000","last_undegraded":"2024-03-14T22:05:23.418804+0000","last_fullsized":"2024-03-14T22:05:23.418804+0000","mapping_epoch":18,"log_start":"23'600","ondisk_log_start":"23'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":92935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'9094","reported_seq":9131,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.421535+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T22:05:17.421535+0000","last_peered":"2024-03-14T22:05:17.421535+0000","last_clean":"2024-03-14T22:05:17.421535+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T22:05:17.421535+0000","last_undegraded":"2024-03-14T22:05:17.421535+0000","last_fullsized":"2024-03-14T22:05:17.421535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9094,"log_dups_size":0,"ondisk_log_size":9094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:57:33.155885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7455,"num_write_kb":104328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9143","reported_seq":9177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.618955+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T22:05:16.618955+0000","last_peered":"2024-03-14T22:05:16.618955+0000","last_clean":"2024-03-14T22:05:16.618955+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T22:05:16.618955+0000","last_undegraded":"2024-03-14T22:05:16.618955+0000","last_fullsized":"2024-03-14T22:05:16.618955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9156,"num_write_kb":60300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11326","reported_seq":11369,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.356317+0000","last_change":"2024-03-14T21:14:03.032533+0000","last_active":"2024-03-14T22:05:23.356317+0000","last_peered":"2024-03-14T22:05:23.356317+0000","last_clean":"2024-03-14T22:05:23.356317+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T22:05:23.356317+0000","last_undegraded":"2024-03-14T22:05:23.356317+0000","last_fullsized":"2024-03-14T22:05:23.356317+0000","mapping_epoch":19,"log_start":"23'1300","ondisk_log_start":"23'1300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:41.581386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2542,"num_write":9699,"num_write_kb":113170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'13351","reported_seq":13939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.410668+0000","last_change":"2024-03-14T21:14:02.659028+0000","last_active":"2024-03-14T22:05:23.410668+0000","last_peered":"2024-03-14T22:05:23.410668+0000","last_clean":"2024-03-14T22:05:23.410668+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T22:05:23.410668+0000","last_undegraded":"2024-03-14T22:05:23.410668+0000","last_fullsized":"2024-03-14T22:05:23.410668+0000","mapping_epoch":18,"log_start":"23'3300","ondisk_log_start":"23'3300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":557,"num_read_kb":4455,"num_write":13349,"num_write_kb":107014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'9102","reported_seq":9148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.342305+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T22:05:17.342305+0000","last_peered":"2024-03-14T22:05:17.342305+0000","last_clean":"2024-03-14T22:05:17.342305+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T22:05:17.342305+0000","last_undegraded":"2024-03-14T22:05:17.342305+0000","last_fullsized":"2024-03-14T22:05:17.342305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9102,"log_dups_size":0,"ondisk_log_size":9102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3318,"num_write":7536,"num_write_kb":95248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'8009","reported_seq":8043,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.437600+0000","last_change":"2024-03-14T21:14:02.663139+0000","last_active":"2024-03-14T22:05:23.437600+0000","last_peered":"2024-03-14T22:05:23.437600+0000","last_clean":"2024-03-14T22:05:23.437600+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T22:05:23.437600+0000","last_undegraded":"2024-03-14T22:05:23.437600+0000","last_fullsized":"2024-03-14T22:05:23.437600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8009,"log_dups_size":0,"ondisk_log_size":8009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:25:31.095452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8008,"num_write_kb":62700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'9642","reported_seq":9681,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.460002+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T22:05:17.460002+0000","last_peered":"2024-03-14T22:05:17.460002+0000","last_clean":"2024-03-14T22:05:17.460002+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T22:05:17.460002+0000","last_undegraded":"2024-03-14T22:05:17.460002+0000","last_fullsized":"2024-03-14T22:05:17.460002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9642,"log_dups_size":0,"ondisk_log_size":9642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8033,"num_write_kb":100651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5756","reported_seq":5791,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.375166+0000","last_change":"2024-03-14T21:14:02.659773+0000","last_active":"2024-03-14T22:05:23.375166+0000","last_peered":"2024-03-14T22:05:23.375166+0000","last_clean":"2024-03-14T22:05:23.375166+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T22:05:23.375166+0000","last_undegraded":"2024-03-14T22:05:23.375166+0000","last_fullsized":"2024-03-14T22:05:23.375166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5756,"log_dups_size":0,"ondisk_log_size":5756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5754,"num_write_kb":48600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'9761","reported_seq":9797,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.410634+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T22:05:17.410634+0000","last_peered":"2024-03-14T22:05:17.410634+0000","last_clean":"2024-03-14T22:05:17.410634+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T22:05:17.410634+0000","last_undegraded":"2024-03-14T22:05:17.410634+0000","last_fullsized":"2024-03-14T22:05:17.410634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8135,"num_write_kb":106503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5851","reported_seq":5886,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.412820+0000","last_change":"2024-03-14T21:14:02.663585+0000","last_active":"2024-03-14T22:05:23.412820+0000","last_peered":"2024-03-14T22:05:23.412820+0000","last_clean":"2024-03-14T22:05:23.412820+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T22:05:23.412820+0000","last_undegraded":"2024-03-14T22:05:23.412820+0000","last_fullsized":"2024-03-14T22:05:23.412820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5851,"log_dups_size":0,"ondisk_log_size":5851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":52920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'9078","reported_seq":9122,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.334095+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T22:05:17.334095+0000","last_peered":"2024-03-14T22:05:17.334095+0000","last_clean":"2024-03-14T22:05:17.334095+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T22:05:17.334095+0000","last_undegraded":"2024-03-14T22:05:17.334095+0000","last_fullsized":"2024-03-14T22:05:17.334095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9078,"log_dups_size":0,"ondisk_log_size":9078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2548,"num_write":7495,"num_write_kb":101746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8321","reported_seq":8356,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.432570+0000","last_change":"2024-03-14T21:14:02.654931+0000","last_active":"2024-03-14T22:05:23.432570+0000","last_peered":"2024-03-14T22:05:23.432570+0000","last_clean":"2024-03-14T22:05:23.432570+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T22:05:23.432570+0000","last_undegraded":"2024-03-14T22:05:23.432570+0000","last_fullsized":"2024-03-14T22:05:23.432570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629345,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":8334,"num_write_kb":67488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'10125","reported_seq":10160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.361153+0000","last_change":"2024-03-14T21:14:44.859568+0000","last_active":"2024-03-14T22:05:17.361153+0000","last_peered":"2024-03-14T22:05:17.361153+0000","last_clean":"2024-03-14T22:05:17.361153+0000","last_became_active":"2024-03-14T21:14:44.859185+0000","last_became_peered":"2024-03-14T21:14:44.859185+0000","last_unstale":"2024-03-14T22:05:17.361153+0000","last_undegraded":"2024-03-14T22:05:17.361153+0000","last_fullsized":"2024-03-14T22:05:17.361153+0000","mapping_epoch":22,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:20:25.220662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":8491,"num_write_kb":113136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"23'8046","reported_seq":8081,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430049+0000","last_change":"2024-03-14T21:14:02.663747+0000","last_active":"2024-03-14T22:05:23.430049+0000","last_peered":"2024-03-14T22:05:23.430049+0000","last_clean":"2024-03-14T22:05:23.430049+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T22:05:23.430049+0000","last_undegraded":"2024-03-14T22:05:23.430049+0000","last_fullsized":"2024-03-14T22:05:23.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:59:11.091842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8051,"num_write_kb":62569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'9547","reported_seq":9589,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.579367+0000","last_change":"2024-03-14T21:14:03.030867+0000","last_active":"2024-03-14T22:05:16.579367+0000","last_peered":"2024-03-14T22:05:16.579367+0000","last_clean":"2024-03-14T22:05:16.579367+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T22:05:16.579367+0000","last_undegraded":"2024-03-14T22:05:16.579367+0000","last_fullsized":"2024-03-14T22:05:16.579367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2042,"num_write":7926,"num_write_kb":109666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'10889","reported_seq":10924,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.438118+0000","last_change":"2024-03-14T21:14:02.659095+0000","last_active":"2024-03-14T22:05:23.438118+0000","last_peered":"2024-03-14T22:05:23.438118+0000","last_clean":"2024-03-14T22:05:23.438118+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T22:05:23.438118+0000","last_undegraded":"2024-03-14T22:05:23.438118+0000","last_fullsized":"2024-03-14T22:05:23.438118+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10896,"num_write_kb":88926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9010","reported_seq":9049,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.599835+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T22:05:16.599835+0000","last_peered":"2024-03-14T22:05:16.599835+0000","last_clean":"2024-03-14T22:05:16.599835+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T22:05:16.599835+0000","last_undegraded":"2024-03-14T22:05:16.599835+0000","last_fullsized":"2024-03-14T22:05:16.599835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9010,"log_dups_size":0,"ondisk_log_size":9010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":7443,"num_write_kb":93234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'11492","reported_seq":11527,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:03:35.733472+0000","last_change":"2024-03-14T21:14:02.663441+0000","last_active":"2024-03-14T22:03:35.733472+0000","last_peered":"2024-03-14T22:03:35.733472+0000","last_clean":"2024-03-14T22:03:35.733472+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T22:03:35.733472+0000","last_undegraded":"2024-03-14T22:03:35.733472+0000","last_fullsized":"2024-03-14T22:03:35.733472+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11486,"num_write_kb":85212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'9510","reported_seq":9550,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.408549+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T22:05:17.408549+0000","last_peered":"2024-03-14T22:05:17.408549+0000","last_clean":"2024-03-14T22:05:17.408549+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T22:05:17.408549+0000","last_undegraded":"2024-03-14T22:05:17.408549+0000","last_fullsized":"2024-03-14T22:05:17.408549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9510,"log_dups_size":0,"ondisk_log_size":9510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":7940,"num_write_kb":115635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8904","reported_seq":8939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421547+0000","last_change":"2024-03-14T21:14:02.653919+0000","last_active":"2024-03-14T22:05:23.421547+0000","last_peered":"2024-03-14T22:05:23.421547+0000","last_clean":"2024-03-14T22:05:23.421547+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T22:05:23.421547+0000","last_undegraded":"2024-03-14T22:05:23.421547+0000","last_fullsized":"2024-03-14T22:05:23.421547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8904,"log_dups_size":0,"ondisk_log_size":8904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8892,"num_write_kb":67382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'9418","reported_seq":9456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.431971+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T22:05:17.431971+0000","last_peered":"2024-03-14T22:05:17.431971+0000","last_clean":"2024-03-14T22:05:17.431971+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T22:05:17.431971+0000","last_undegraded":"2024-03-14T22:05:17.431971+0000","last_fullsized":"2024-03-14T22:05:17.431971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9418,"log_dups_size":0,"ondisk_log_size":9418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7820,"num_write_kb":110543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:02:10.082485+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T22:02:10.082485+0000","last_peered":"2024-03-14T22:02:10.082485+0000","last_clean":"2024-03-14T22:02:10.082485+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T22:02:10.082485+0000","last_undegraded":"2024-03-14T22:02:10.082485+0000","last_fullsized":"2024-03-14T22:02:10.082485+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'8577","reported_seq":8614,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430551+0000","last_change":"2024-03-14T21:14:02.652881+0000","last_active":"2024-03-14T22:05:23.430551+0000","last_peered":"2024-03-14T22:05:23.430551+0000","last_clean":"2024-03-14T22:05:23.430551+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T22:05:23.430551+0000","last_undegraded":"2024-03-14T22:05:23.430551+0000","last_fullsized":"2024-03-14T22:05:23.430551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8577,"log_dups_size":0,"ondisk_log_size":8577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":67724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'11125","reported_seq":11164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.609325+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T22:05:16.609325+0000","last_peered":"2024-03-14T22:05:16.609325+0000","last_clean":"2024-03-14T22:05:16.609325+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T22:05:16.609325+0000","last_undegraded":"2024-03-14T22:05:16.609325+0000","last_fullsized":"2024-03-14T22:05:16.609325+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":9507,"num_write_kb":117178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'8676","reported_seq":8714,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421927+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T22:05:23.421927+0000","last_peered":"2024-03-14T22:05:23.421927+0000","last_clean":"2024-03-14T22:05:23.421927+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T22:05:23.421927+0000","last_undegraded":"2024-03-14T22:05:23.421927+0000","last_fullsized":"2024-03-14T22:05:23.421927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8676,"log_dups_size":0,"ondisk_log_size":8676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:46.126855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8672,"num_write_kb":67194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10241","reported_seq":10279,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.608108+0000","last_change":"2024-03-14T21:14:03.033195+0000","last_active":"2024-03-14T22:05:16.608108+0000","last_peered":"2024-03-14T22:05:16.608108+0000","last_clean":"2024-03-14T22:05:16.608108+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T22:05:16.608108+0000","last_undegraded":"2024-03-14T22:05:16.608108+0000","last_fullsized":"2024-03-14T22:05:16.608108+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8656,"num_write_kb":115905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11303","reported_seq":12007,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.404838+0000","last_change":"2024-03-14T21:14:02.658709+0000","last_active":"2024-03-14T22:05:23.404838+0000","last_peered":"2024-03-14T22:05:23.404838+0000","last_clean":"2024-03-14T22:05:23.404838+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T22:05:23.404838+0000","last_undegraded":"2024-03-14T22:05:23.404838+0000","last_fullsized":"2024-03-14T22:05:23.404838+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":669,"num_read_kb":4516,"num_write":11296,"num_write_kb":89038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'9435","reported_seq":9475,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.585185+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T22:05:16.585185+0000","last_peered":"2024-03-14T22:05:16.585185+0000","last_clean":"2024-03-14T22:05:16.585185+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T22:05:16.585185+0000","last_undegraded":"2024-03-14T22:05:16.585185+0000","last_fullsized":"2024-03-14T22:05:16.585185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9435,"log_dups_size":0,"ondisk_log_size":9435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7837,"num_write_kb":123513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'9296","reported_seq":9333,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.408089+0000","last_change":"2024-03-14T21:14:02.657401+0000","last_active":"2024-03-14T22:05:23.408089+0000","last_peered":"2024-03-14T22:05:23.408089+0000","last_clean":"2024-03-14T22:05:23.408089+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T22:05:23.408089+0000","last_undegraded":"2024-03-14T22:05:23.408089+0000","last_fullsized":"2024-03-14T22:05:23.408089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9294,"num_write_kb":71830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'9543","reported_seq":9580,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.459567+0000","last_change":"2024-03-14T21:14:03.036152+0000","last_active":"2024-03-14T22:05:17.459567+0000","last_peered":"2024-03-14T22:05:17.459567+0000","last_clean":"2024-03-14T22:05:17.459567+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T22:05:17.459567+0000","last_undegraded":"2024-03-14T22:05:17.459567+0000","last_fullsized":"2024-03-14T22:05:17.459567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9543,"log_dups_size":0,"ondisk_log_size":9543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:14.794102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":7936,"num_write_kb":109238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'11129","reported_seq":11166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.404373+0000","last_change":"2024-03-14T21:14:02.653065+0000","last_active":"2024-03-14T22:05:17.404373+0000","last_peered":"2024-03-14T22:05:17.404373+0000","last_clean":"2024-03-14T22:05:17.404373+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T22:05:17.404373+0000","last_undegraded":"2024-03-14T22:05:17.404373+0000","last_fullsized":"2024-03-14T22:05:17.404373+0000","mapping_epoch":18,"log_start":"23'1100","ondisk_log_start":"23'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11129,"num_write_kb":85841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'12164","reported_seq":12200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.438275+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T22:05:17.438275+0000","last_peered":"2024-03-14T22:05:17.438275+0000","last_clean":"2024-03-14T22:05:17.438275+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T22:05:17.438275+0000","last_undegraded":"2024-03-14T22:05:17.438275+0000","last_fullsized":"2024-03-14T22:05:17.438275+0000","mapping_epoch":19,"log_start":"23'2100","ondisk_log_start":"23'2100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10560,"num_write_kb":124816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'9283","reported_seq":9320,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.431083+0000","last_change":"2024-03-14T21:14:02.659612+0000","last_active":"2024-03-14T22:05:23.431083+0000","last_peered":"2024-03-14T22:05:23.431083+0000","last_clean":"2024-03-14T22:05:23.431083+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T22:05:23.431083+0000","last_undegraded":"2024-03-14T22:05:23.431083+0000","last_fullsized":"2024-03-14T22:05:23.431083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9283,"log_dups_size":0,"ondisk_log_size":9283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9294,"num_write_kb":71728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9769","reported_seq":9810,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.417912+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T22:05:17.417912+0000","last_peered":"2024-03-14T22:05:17.417912+0000","last_clean":"2024-03-14T22:05:17.417912+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T22:05:17.417912+0000","last_undegraded":"2024-03-14T22:05:17.417912+0000","last_fullsized":"2024-03-14T22:05:17.417912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9769,"log_dups_size":0,"ondisk_log_size":9769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8091,"num_write_kb":107879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'8352","reported_seq":8981,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:11.741680+0000","last_change":"2024-03-14T21:14:02.662122+0000","last_active":"2024-03-14T22:05:11.741680+0000","last_peered":"2024-03-14T22:05:11.741680+0000","last_clean":"2024-03-14T22:05:11.741680+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T22:05:11.741680+0000","last_undegraded":"2024-03-14T22:05:11.741680+0000","last_fullsized":"2024-03-14T22:05:11.741680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8352,"log_dups_size":0,"ondisk_log_size":8352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":593,"num_read_kb":4491,"num_write":9527,"num_write_kb":73065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'10070","reported_seq":10114,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.350964+0000","last_change":"2024-03-14T21:14:03.037921+0000","last_active":"2024-03-14T22:05:23.350964+0000","last_peered":"2024-03-14T22:05:23.350964+0000","last_clean":"2024-03-14T22:05:23.350964+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T22:05:23.350964+0000","last_undegraded":"2024-03-14T22:05:23.350964+0000","last_fullsized":"2024-03-14T22:05:23.350964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8532,"num_write_kb":114847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5880","reported_seq":5916,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.420296+0000","last_change":"2024-03-14T21:14:02.665745+0000","last_active":"2024-03-14T22:05:23.420296+0000","last_peered":"2024-03-14T22:05:23.420296+0000","last_clean":"2024-03-14T22:05:23.420296+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T22:05:23.420296+0000","last_undegraded":"2024-03-14T22:05:23.420296+0000","last_fullsized":"2024-03-14T22:05:23.420296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5880,"log_dups_size":0,"ondisk_log_size":5880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5885,"num_write_kb":44672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'10911","reported_seq":10953,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.453069+0000","last_change":"2024-03-14T21:14:03.029269+0000","last_active":"2024-03-14T22:05:17.453069+0000","last_peered":"2024-03-14T22:05:17.453069+0000","last_clean":"2024-03-14T22:05:17.453069+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T22:05:17.453069+0000","last_undegraded":"2024-03-14T22:05:17.453069+0000","last_fullsized":"2024-03-14T22:05:17.453069+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":9274,"num_write_kb":112928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'4360","reported_seq":4396,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.551031+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T22:05:15.551031+0000","last_peered":"2024-03-14T22:05:15.551031+0000","last_clean":"2024-03-14T22:05:15.551031+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T22:05:15.551031+0000","last_undegraded":"2024-03-14T22:05:15.551031+0000","last_fullsized":"2024-03-14T22:05:15.551031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4362,"num_write_kb":43693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'9883","reported_seq":9931,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.440163+0000","last_change":"2024-03-14T21:14:03.039549+0000","last_active":"2024-03-14T22:05:16.440163+0000","last_peered":"2024-03-14T22:05:16.440163+0000","last_clean":"2024-03-14T22:05:16.440163+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T22:05:16.440163+0000","last_undegraded":"2024-03-14T22:05:16.440163+0000","last_fullsized":"2024-03-14T22:05:16.440163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9883,"log_dups_size":0,"ondisk_log_size":9883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:23.510516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":8266,"num_write_kb":116331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'6399","reported_seq":6435,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.600135+0000","last_change":"2024-03-14T21:14:02.660292+0000","last_active":"2024-03-14T22:05:16.600135+0000","last_peered":"2024-03-14T22:05:16.600135+0000","last_clean":"2024-03-14T22:05:16.600135+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T22:05:16.600135+0000","last_undegraded":"2024-03-14T22:05:16.600135+0000","last_fullsized":"2024-03-14T22:05:16.600135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6399,"log_dups_size":0,"ondisk_log_size":6399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6417,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'10919","reported_seq":10954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.435069+0000","last_change":"2024-03-14T21:14:03.040817+0000","last_active":"2024-03-14T22:05:17.435069+0000","last_peered":"2024-03-14T22:05:17.435069+0000","last_clean":"2024-03-14T22:05:17.435069+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T22:05:17.435069+0000","last_undegraded":"2024-03-14T22:05:17.435069+0000","last_fullsized":"2024-03-14T22:05:17.435069+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":120976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'9349","reported_seq":9385,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:01:50.712636+0000","last_change":"2024-03-14T21:14:02.661073+0000","last_active":"2024-03-14T22:01:50.712636+0000","last_peered":"2024-03-14T22:01:50.712636+0000","last_clean":"2024-03-14T22:01:50.712636+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T22:01:50.712636+0000","last_undegraded":"2024-03-14T22:01:50.712636+0000","last_fullsized":"2024-03-14T22:01:50.712636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9349,"log_dups_size":0,"ondisk_log_size":9349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9372,"num_write_kb":71713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'10191","reported_seq":10229,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.350982+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T22:05:17.350982+0000","last_peered":"2024-03-14T22:05:17.350982+0000","last_clean":"2024-03-14T22:05:17.350982+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T22:05:17.350982+0000","last_undegraded":"2024-03-14T22:05:17.350982+0000","last_fullsized":"2024-03-14T22:05:17.350982+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8519,"num_write_kb":106953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'11400","reported_seq":11835,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.412968+0000","last_change":"2024-03-14T21:14:02.663131+0000","last_active":"2024-03-14T22:05:23.412968+0000","last_peered":"2024-03-14T22:05:23.412968+0000","last_clean":"2024-03-14T22:05:23.412968+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T22:05:23.412968+0000","last_undegraded":"2024-03-14T22:05:23.412968+0000","last_fullsized":"2024-03-14T22:05:23.412968+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13689487,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":401,"num_read_kb":1362,"num_write":11406,"num_write_kb":86543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'13457","reported_seq":13498,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.396876+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T22:05:17.396876+0000","last_peered":"2024-03-14T22:05:17.396876+0000","last_clean":"2024-03-14T22:05:17.396876+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T22:05:17.396876+0000","last_undegraded":"2024-03-14T22:05:17.396876+0000","last_fullsized":"2024-03-14T22:05:17.396876+0000","mapping_epoch":19,"log_start":"23'3400","ondisk_log_start":"23'3400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":11847,"num_write_kb":149794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'10672","reported_seq":10705,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:16.469190+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T22:05:16.469190+0000","last_peered":"2024-03-14T22:05:16.469190+0000","last_clean":"2024-03-14T22:05:16.469190+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T22:05:16.469190+0000","last_undegraded":"2024-03-14T22:05:16.469190+0000","last_fullsized":"2024-03-14T22:05:16.469190+0000","mapping_epoch":19,"log_start":"23'600","ondisk_log_start":"23'600","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":8986,"num_write_kb":111494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6864","reported_seq":6897,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.409531+0000","last_change":"2024-03-14T21:14:02.660393+0000","last_active":"2024-03-14T22:05:23.409531+0000","last_peered":"2024-03-14T22:05:23.409531+0000","last_clean":"2024-03-14T22:05:23.409531+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T22:05:23.409531+0000","last_undegraded":"2024-03-14T22:05:23.409531+0000","last_fullsized":"2024-03-14T22:05:23.409531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6864,"log_dups_size":0,"ondisk_log_size":6864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6885,"num_write_kb":66562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'11241","reported_seq":11278,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.351892+0000","last_change":"2024-03-14T21:14:44.859446+0000","last_active":"2024-03-14T22:05:17.351892+0000","last_peered":"2024-03-14T22:05:17.351892+0000","last_clean":"2024-03-14T22:05:17.351892+0000","last_became_active":"2024-03-14T21:14:44.859128+0000","last_became_peered":"2024-03-14T21:14:44.859128+0000","last_unstale":"2024-03-14T22:05:17.351892+0000","last_undegraded":"2024-03-14T22:05:17.351892+0000","last_fullsized":"2024-03-14T22:05:17.351892+0000","mapping_epoch":22,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:32:11.377675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1780,"num_write":9572,"num_write_kb":116659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15422","reported_seq":16524,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.386679+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T22:05:23.386679+0000","last_peered":"2024-03-14T22:05:23.386679+0000","last_clean":"2024-03-14T22:05:23.386679+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T22:05:23.386679+0000","last_undegraded":"2024-03-14T22:05:23.386679+0000","last_fullsized":"2024-03-14T22:05:23.386679+0000","mapping_epoch":18,"log_start":"23'5400","ondisk_log_start":"23'5400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:48.203059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1069,"num_read_kb":4840,"num_write":15404,"num_write_kb":112501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'10425","reported_seq":10472,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.447186+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T22:05:17.447186+0000","last_peered":"2024-03-14T22:05:17.447186+0000","last_clean":"2024-03-14T22:05:17.447186+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T22:05:17.447186+0000","last_undegraded":"2024-03-14T22:05:17.447186+0000","last_fullsized":"2024-03-14T22:05:17.447186+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2796,"num_write":8857,"num_write_kb":109660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'8706","reported_seq":8744,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.407645+0000","last_change":"2024-03-14T21:14:02.654963+0000","last_active":"2024-03-14T22:05:23.407645+0000","last_peered":"2024-03-14T22:05:23.407645+0000","last_clean":"2024-03-14T22:05:23.407645+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T22:05:23.407645+0000","last_undegraded":"2024-03-14T22:05:23.407645+0000","last_fullsized":"2024-03-14T22:05:23.407645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8706,"log_dups_size":0,"ondisk_log_size":8706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8698,"num_write_kb":67327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'11830","reported_seq":11868,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.429804+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T22:05:17.429804+0000","last_peered":"2024-03-14T22:05:17.429804+0000","last_clean":"2024-03-14T22:05:17.429804+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T22:05:17.429804+0000","last_undegraded":"2024-03-14T22:05:17.429804+0000","last_fullsized":"2024-03-14T22:05:17.429804+0000","mapping_epoch":19,"log_start":"23'1800","ondisk_log_start":"23'1800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":10179,"num_write_kb":107836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6046","reported_seq":6083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.408036+0000","last_change":"2024-03-14T21:14:02.656125+0000","last_active":"2024-03-14T22:05:23.408036+0000","last_peered":"2024-03-14T22:05:23.408036+0000","last_clean":"2024-03-14T22:05:23.408036+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T22:05:23.408036+0000","last_undegraded":"2024-03-14T22:05:23.408036+0000","last_fullsized":"2024-03-14T22:05:23.408036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6048,"num_write_kb":48854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2628,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":345,"num_read_kb":87814,"num_write":552755,"num_write_kb":7123402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624176,"ondisk_log_size":624176,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":256812718,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":3940,"num_read_kb":24158,"num_write":290939,"num_write_kb":2282021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":513675264,"data_stored":513612702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271768,"ondisk_log_size":271768,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575471,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252664,"kb_used_data":112496,"kb_used_omap":1,"kb_used_meta":1140158,"kb_avail":93119176,"statfs":{"total":96636764160,"available":95354036224,"internally_reserved":0,"allocated":115195904,"data_stored":115031492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1167522059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45500000000000002,"15min":0.48699999999999999},"min":{"1min":0.26100000000000001,"5min":0.23000000000000001,"15min":0.19500000000000001},"max":{"1min":0.57499999999999996,"5min":2.782,"15min":4.7999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.63900000000000001,"5min":2.8519999999999999,"15min":3.9239999999999999},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":0.66000000000000003,"5min":1.8520000000000001,"15min":3.903},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51100000000000001,"15min":0.55900000000000005},"min":{"1min":0.30099999999999999,"5min":0.28599999999999998,"15min":0.26400000000000001},"max":{"1min":0.60599999999999998,"5min":6.1440000000000001,"15min":6.6689999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":0.63500000000000001,"5min":3.8210000000000002,"15min":4.8310000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.59199999999999997,"15min":0.54700000000000004},"min":{"1min":0.32000000000000001,"5min":0.29799999999999999,"15min":0.25900000000000001},"max":{"1min":0.59799999999999998,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67200000000000004,"5min":6.1820000000000004,"15min":6.6909999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.5,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.622,"5min":1.9339999999999999,"15min":4.851},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.68700000000000006,"5min":1.226,"15min":6.7220000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.44700000000000001},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.23300000000000001},"max":{"1min":0.69899999999999995,"5min":1.492,"15min":5.2679999999999998},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42999999999999999,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.255,"15min":0.24199999999999999},"max":{"1min":0.71799999999999997,"5min":1.3089999999999999,"15min":3.8740000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.755,"5min":1.321,"15min":6.6459999999999999},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.49399999999999999},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69599999999999995,"5min":1.2529999999999999,"15min":4.157},"last":0.442},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.55800000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":1.2509999999999999,"15min":6.6109999999999998},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575470,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101820,"kb_used_data":69876,"kb_used_omap":1,"kb_used_meta":1031934,"kb_avail":93270020,"statfs":{"total":96636764160,"available":95508500480,"internally_reserved":0,"allocated":71553024,"data_stored":71391599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1056700683},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.47299999999999998,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.188},"max":{"1min":2.899,"5min":2.899,"15min":17.885999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.23000000000000001},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.52000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.216},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":7.226},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":1.224,"15min":7.1699999999999999},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.6189999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.26800000000000002},"max":{"1min":3.282,"5min":3.282,"15min":7.2489999999999997},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":1.2470000000000001,"15min":7.1470000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.61799999999999999,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.29899999999999999},"max":{"1min":6.1660000000000004,"5min":6.1660000000000004,"15min":6.1660000000000004},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.468,"5min":2.468,"15min":7.3330000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.49199999999999999,"15min":0.45300000000000001},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":6.0229999999999997,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":8.234,"5min":8.234,"15min":8.234},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48799999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.90600000000000003,"5min":1.3340000000000001,"15min":7.5380000000000003},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.54100000000000004,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":7.2039999999999997},"last":0.80800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575470,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1147640,"kb_used_data":69872,"kb_used_omap":1,"kb_used_meta":1077758,"kb_avail":93224200,"statfs":{"total":96636764160,"available":95461580800,"internally_reserved":0,"allocated":71548928,"data_stored":71391017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1103624459},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.55500000000000005},"min":{"1min":0.27300000000000002,"5min":0.221,"15min":0.192},"max":{"1min":0.71199999999999997,"5min":3.2149999999999999,"15min":7.0990000000000002},"last":0.438},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.57499999999999996},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.21099999999999999},"max":{"1min":0.66800000000000004,"5min":5.7359999999999998,"15min":12.281000000000001},"last":0.318}]},{"osd":1,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.68100000000000005,"15min":0.60399999999999998},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.20599999999999999},"max":{"1min":0.70299999999999996,"5min":14.952,"15min":14.952},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49299999999999999,"15min":0.54500000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.222},"max":{"1min":0.63600000000000001,"5min":3.1339999999999999,"15min":8.8539999999999992},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.75800000000000001,"15min":0.63100000000000001},"min":{"1min":0.28699999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":0.70599999999999996,"5min":18.623000000000001,"15min":18.623000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.96399999999999997,"15min":0.70699999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.67000000000000004,"5min":30.004999999999999,"15min":30.004999999999999},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.25},"max":{"1min":0.66200000000000003,"5min":3.6600000000000001,"15min":9.2309999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.57699999999999996},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":0.82699999999999996,"15min":7.3780000000000001},"last":0.498}]},{"osd":4,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.59899999999999998,"15min":0.53700000000000003},"min":{"1min":0.22700000000000001,"5min":0.219,"15min":0.20999999999999999},"max":{"1min":0.58399999999999996,"5min":12.414,"15min":12.414},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.496,"15min":0.57099999999999995},"min":{"1min":0.308,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60799999999999998,"5min":3.1970000000000001,"15min":8.9220000000000006},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.44600000000000001,"15min":0.46600000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.22},"max":{"1min":0.55600000000000005,"5min":1.905,"15min":7.4000000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.61599999999999999,"15min":0.58299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.24299999999999999},"max":{"1min":0.65700000000000003,"5min":10.278,"15min":10.278},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.52300000000000002,"15min":0.55100000000000005},"min":{"1min":0.27500000000000002,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":0.64200000000000002,"5min":3.9820000000000002,"15min":7.1619999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.28399999999999997},"max":{"1min":0.623,"5min":3.7789999999999999,"15min":7.3310000000000004},"last":0.57199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":961852,"kb_used_data":42292,"kb_used_omap":1,"kb_used_meta":919550,"kb_avail":93409988,"statfs":{"total":96636764160,"available":95651827712,"internally_reserved":0,"allocated":43307008,"data_stored":43137500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":941619467},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.495,"15min":0.55600000000000005},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":0.73799999999999999,"5min":2.23,"15min":15.013},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.26000000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.72599999999999998,"5min":2.1139999999999999,"15min":5.6879999999999997},"last":0.628}]},{"osd":1,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":1.806,"15min":4.6479999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.52000000000000002},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":0.64900000000000002,"5min":0.77800000000000002,"15min":4.343},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.503},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70699999999999996,"5min":0.97099999999999997,"15min":5.5540000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57499999999999996,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.26000000000000001},"max":{"1min":0.75700000000000001,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.5}]},{"osd":3,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.22},"max":{"1min":0.67700000000000005,"5min":2.1320000000000001,"15min":9.9800000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.33000000000000002,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":0.77200000000000002,"5min":1.8919999999999999,"15min":5.899},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41499999999999998,"15min":0.40999999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.20599999999999999},"max":{"1min":0.61099999999999999,"5min":0.71599999999999997,"15min":3.0470000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.41899999999999998,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.61699999999999999,"5min":0.69999999999999996,"15min":4.2460000000000004},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44500000000000001,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.76100000000000001,"5min":0.90900000000000003,"15min":3.8650000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45700000000000002,"15min":0.45900000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":1.5680000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.23400000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":8.7639999999999993},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.27400000000000002},"max":{"1min":0.69399999999999995,"5min":0.94799999999999995,"15min":3.7850000000000001},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1131324,"kb_used_data":57588,"kb_used_omap":1,"kb_used_meta":1073726,"kb_avail":93240516,"statfs":{"total":96636764160,"available":95478288384,"internally_reserved":0,"allocated":58970112,"data_stored":58808687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1099495691},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.57599999999999996,"15min":0.53000000000000003},"min":{"1min":0.23000000000000001,"5min":0.20499999999999999,"15min":0.17000000000000001},"max":{"1min":0.52300000000000002,"5min":7.7270000000000003,"15min":14.695},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.186},"max":{"1min":0.51900000000000002,"5min":7.9770000000000003,"15min":8.9649999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.378,"15min":0.45500000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":0.71799999999999997,"5min":0.93500000000000005,"15min":5.5289999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.53700000000000003,"15min":0.60699999999999998},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.20100000000000001},"max":{"1min":0.83899999999999997,"5min":6.1740000000000004,"15min":14.898},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.61599999999999999,"15min":0.57099999999999995},"min":{"1min":0.251,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.83099999999999996,"5min":9.4079999999999995,"15min":9.5150000000000006},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.55900000000000005},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":2.9430000000000001,"15min":15.263},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.71899999999999997,"15min":0.74399999999999999},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":0.77700000000000002,"5min":9.6080000000000005,"15min":14.071999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.622,"15min":0.69499999999999995},"min":{"1min":0.40200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":3.5710000000000002,"15min":14.109},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.70899999999999996,"15min":0.68200000000000005},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":11.443,"15min":11.443},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.74199999999999999,"15min":0.77000000000000002},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.78000000000000003,"5min":7.6399999999999997,"15min":14.550000000000001},"last":0.85399999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.82999999999999996,"15min":0.76500000000000001},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.29299999999999998},"max":{"1min":0.78400000000000003,"5min":9.7200000000000006,"15min":9.7200000000000006},"last":0.873},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.78200000000000003,"15min":0.69899999999999995},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":15.339,"15min":15.339},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.81599999999999995,"15min":0.77700000000000002},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":7.7030000000000003,"15min":14.726000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.83399999999999996,"15min":0.76300000000000001},"min":{"1min":0.433,"5min":0.379,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":9.5079999999999991,"15min":9.5079999999999991},"last":0.91400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575471,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649700,"kb_used_data":45148,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":93722140,"statfs":{"total":96636764160,"available":95971471360,"internally_reserved":0,"allocated":46231552,"data_stored":46077232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":619051275},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.61299999999999999,"15min":0.45800000000000002},"min":{"1min":0.24399999999999999,"5min":0.184,"15min":0.182},"max":{"1min":2.347,"5min":10.869,"15min":10.869},"last":0.311},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.67800000000000005,"15min":0.55200000000000005},"min":{"1min":0.23799999999999999,"5min":0.20100000000000001,"15min":0.189},"max":{"1min":1.107,"5min":9.6649999999999991,"15min":12.317},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.53100000000000003,"15min":0.45600000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":2.0049999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.20100000000000001},"max":{"1min":0.96799999999999997,"5min":7.8090000000000002,"15min":9.5429999999999993},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.66700000000000004,"15min":0.61899999999999999},"min":{"1min":0.26500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56499999999999995,"5min":13.130000000000001,"15min":27.355},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52800000000000002,"15min":0.57099999999999995},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.26000000000000001},"max":{"1min":1.929,"5min":3.4390000000000001,"15min":12.451000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83099999999999996,"15min":0.71999999999999997},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.28899999999999998},"max":{"1min":7.1310000000000002,"5min":8.0440000000000005,"15min":12.478999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.79800000000000004,"15min":0.73699999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.30599999999999999},"max":{"1min":0.86899999999999999,"5min":7.9409999999999998,"15min":12.391999999999999},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.0529999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":0.97299999999999998,"5min":20.84,"15min":20.84},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69199999999999995,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.0329999999999999,"5min":4.7240000000000002,"15min":11.762},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.69699999999999995,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.79800000000000004,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.78000000000000003,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.9590000000000001,"5min":13.177,"15min":13.177},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.79400000000000004,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.35099999999999998,"15min":0.32600000000000001},"max":{"1min":1.8420000000000001,"5min":6.2309999999999999,"15min":11.865},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1031156,"kb_used_data":57452,"kb_used_omap":1,"kb_used_meta":973694,"kb_avail":93340684,"statfs":{"total":96636764160,"available":95580860416,"internally_reserved":0,"allocated":58830848,"data_stored":58655239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":997062923},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49399999999999999,"15min":0.439},"min":{"1min":0.19400000000000001,"5min":0.186,"15min":0.184},"max":{"1min":0.89000000000000001,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.219,"5min":0.219,"15min":0.19700000000000001},"max":{"1min":2.145,"5min":6.5339999999999998,"15min":9.2249999999999996},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54800000000000004,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":2.161,"5min":13.553000000000001,"15min":13.553000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":2.0910000000000002,"5min":6.6050000000000004,"15min":7.1180000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.70699999999999996,"15min":0.64000000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.22900000000000001},"max":{"1min":2.1190000000000002,"5min":10.236000000000001,"15min":24.879000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.53400000000000003},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.22900000000000001},"max":{"1min":1.4670000000000001,"5min":2.2290000000000001,"15min":20.995000000000001},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":7.1609999999999996,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":8.4420000000000002,"5min":8.4420000000000002,"15min":8.4420000000000002},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9059999999999999,"5min":3.5800000000000001,"15min":8.6419999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.79100000000000004,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.96699999999999997,"5min":16.943999999999999,"15min":16.943999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.0270000000000001,"5min":3.5409999999999999,"15min":7.468},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1,"5min":3.4950000000000001,"15min":5.5030000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":2.5169999999999999,"5min":8.593,"15min":9.3209999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.40200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.496,"5min":6.7859999999999996,"15min":10.346},"last":0.80400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":871104,"kb_used_data":49976,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":93500736,"statfs":{"total":96636764160,"available":95744753664,"internally_reserved":0,"allocated":51175424,"data_stored":51009794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":840825099},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.45500000000000002},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.17499999999999999},"max":{"1min":2.8149999999999999,"5min":12.734,"15min":12.734},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52000000000000002,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":3.976,"5min":9.3620000000000001,"15min":9.3620000000000001},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.843,"5min":15.375,"15min":15.375},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.48099999999999998},"min":{"1min":0.28399999999999997,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.008,"5min":9.2959999999999994,"15min":9.2959999999999994},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.20699999999999999},"max":{"1min":2.7040000000000002,"5min":8.7230000000000008,"15min":8.7230000000000008},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":2.9169999999999998,"5min":10.928000000000001,"15min":12.864000000000001},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0350000000000001,"5min":9.2319999999999993,"15min":9.2319999999999993},"last":0.435},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":3.7930000000000001,"15min":7.5499999999999998},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.61899999999999999},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":3.0790000000000002,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.68700000000000006},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"max":{"1min":2.8660000000000001,"5min":10.135999999999999,"15min":11.068},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.69699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.8959999999999999,"5min":12.77,"15min":12.77},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":4.0579999999999998,"5min":9.391,"15min":9.391},"last":0.375}]},{"osd":7,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73299999999999998,"15min":0.67900000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.0830000000000002,"5min":9.3290000000000006,"15min":9.3290000000000006},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.71499999999999997,"15min":0.65400000000000003},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.33300000000000002},"max":{"1min":0.78900000000000003,"5min":7.8739999999999997,"15min":7.8739999999999997},"last":0.47399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58568704,"data_stored":58559811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45989888,"data_stored":45989473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43065344,"data_stored":43049741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":71307264,"data_stored":71303258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":71311360,"data_stored":71303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114360320,"data_stored":114353365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":7669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T22:05:27.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T22:05:27.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.549+0000 7fb7064ea640 1 Processor -- start 2024-03-14T22:05:27.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.549+0000 7fb7064ea640 1 -- start start 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb7064ea640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb70006ba90 0x7fb7000bbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb7064ea640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7000bfbc0 con 0x7fb70006a950 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb7064ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7000bc950 con 0x7fb7000a5250 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb7064ea640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7000bf670 con 0x7fb70006ba90 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb6ff7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb70006ba90 0x7fb7000bbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb6fffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb70006a950 0x7fb70014ac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43576/0 (socket says 172.21.15.49:43576) 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb6fffff640 1 -- 172.21.15.49:0/99803689 learned_addr learned my addr 172.21.15.49:0/99803689 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb6ff7fe640 1 -- 172.21.15.49:0/99803689 >> v1:172.21.15.97:6789/0 conn(0x7fb7000a5250 legacy=0x7fb700155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb6ff7fe640 1 -- 172.21.15.49:0/99803689 >> v1:172.21.15.49:6789/0 conn(0x7fb70006a950 legacy=0x7fb70014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb6feffd640 1 -- 172.21.15.49:0/99803689 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277782628 0 0) 0x7fb7000bfbc0 con 0x7fb70006a950 2024-03-14T22:05:27.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.550+0000 7fb6ff7fe640 1 -- 172.21.15.49:0/99803689 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000c0050 con 0x7fb70006ba90 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb6ff7fe640 1 --2- 172.21.15.49:0/99803689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006ba90 0x7fb7000bbd10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00097a0 tx=0x7fb6f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce9e1f98a406b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb6feffd640 1 -- 172.21.15.49:0/99803689 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f002e070 con 0x7fb70006ba90 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb6feffd640 1 -- 172.21.15.49:0/99803689 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f00315a0 con 0x7fb70006ba90 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb6feffd640 1 -- 172.21.15.49:0/99803689 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0031930 con 0x7fb70006ba90 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb7064ea640 1 -- 172.21.15.49:0/99803689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006ba90 msgr2=0x7fb7000bbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb7064ea640 1 --2- 172.21.15.49:0/99803689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006ba90 0x7fb7000bbd10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00097a0 tx=0x7fb6f002b840 comp rx=0 tx=0).stop 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb7064ea640 1 -- 172.21.15.49:0/99803689 shutdown_connections 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb7064ea640 1 --2- 172.21.15.49:0/99803689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb70006ba90 0x7fb7000bbd10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.551+0000 7fb7064ea640 1 -- 172.21.15.49:0/99803689 >> 172.21.15.49:0/99803689 conn(0x7fb7000c0460 msgr2=0x7fb7000c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.552+0000 7fb7064ea640 1 -- 172.21.15.49:0/99803689 shutdown_connections 2024-03-14T22:05:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.552+0000 7fb7064ea640 1 -- 172.21.15.49:0/99803689 wait complete. 2024-03-14T22:05:27.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 Processor -- start 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 -- start start 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb70006a950 0x7fb7000b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7000a5250 0x7fb7000b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7000b9a20 0x7fb7000b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000b4240 con 0x7fb7000b9a20 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000b69b0 con 0x7fb70006a950 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb7064ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000b4440 con 0x7fb7000a5250 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.553+0000 7fb704a87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7000b9a20 0x7fb7000b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7000b9a20 0x7fb7000b9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50164/0 (socket says 172.21.15.49:50164) 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 -- 172.21.15.49:0/9928652 learned_addr learned my addr 172.21.15.49:0/9928652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb6fffff640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb70006a950 0x7fb7000b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb6ff7fe640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7000a5250 0x7fb7000b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 -- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7000a5250 msgr2=0x7fb7000b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7000a5250 0x7fb7000b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 -- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb70006a950 msgr2=0x7fb7000b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb70006a950 0x7fb7000b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 -- 172.21.15.49:0/9928652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0009400 con 0x7fb7000b9a20 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb704a87640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7000b9a20 0x7fb7000b9eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800c6b0 tx=0x7fb6f800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb6fcff9640 1 -- 172.21.15.49:0/9928652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8003b70 con 0x7fb7000b9a20 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb6fcff9640 1 -- 172.21.15.49:0/9928652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f8003cf0 con 0x7fb7000b9a20 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7000b4720 con 0x7fb7000b9a20 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb6fcff9640 1 -- 172.21.15.49:0/9928652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8003ea0 con 0x7fb7000b9a20 2024-03-14T22:05:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.554+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7000b4ce0 con 0x7fb7000b9a20 2024-03-14T22:05:27.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.555+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7000674d0 con 0x7fb7000b9a20 2024-03-14T22:05:27.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.556+0000 7fb6fcff9640 1 -- 172.21.15.49:0/9928652 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb6f8005310 con 0x7fb7000b9a20 2024-03-14T22:05:27.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.556+0000 7fb6fcff9640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb6ec06a7a0 0x7fb6ec06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:27.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.556+0000 7fb6fcff9640 1 -- 172.21.15.49:0/9928652 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb6f8088b50 con 0x7fb7000b9a20 2024-03-14T22:05:27.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.556+0000 7fb6fffff640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb6ec06a7a0 0x7fb6ec06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:27.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.557+0000 7fb6fffff640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb6ec06a7a0 0x7fb6ec06cc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4000f60 tx=0x7fb6f4007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:27.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.560+0000 7fb6fcff9640 1 -- 172.21.15.49:0/9928652 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb6f8053f80 con 0x7fb7000b9a20 2024-03-14T22:05:27.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.740+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb700066bb0 con 0x7fb7000b9a20 2024-03-14T22:05:27.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.741+0000 7fb6fcff9640 1 -- 172.21.15.49:0/9928652 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16239 (secure 0 0 0) 0x7fb6f80575a0 con 0x7fb7000b9a20 2024-03-14T22:05:27.740 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:05:27.740 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:44.734113+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb6ec06a7a0 msgr2=0x7fb6ec06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb6ec06a7a0 0x7fb6ec06cc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4000f60 tx=0x7fb6f4007b90 comp rx=0 tx=0).stop 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7000b9a20 msgr2=0x7fb7000b9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7000b9a20 0x7fb7000b9eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800c6b0 tx=0x7fb6f800cb70 comp rx=0 tx=0).stop 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 shutdown_connections 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fb6ec06a7a0 0x7fb6ec06cc60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7000b9a20 0x7fb7000b9eb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7000a5250 0x7fb7000b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 --2- 172.21.15.49:0/9928652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb70006a950 0x7fb7000b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 >> 172.21.15.49:0/9928652 conn(0x7fb7000c0460 msgr2=0x7fb7000a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 shutdown_connections 2024-03-14T22:05:27.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:27.743+0000 7fb7064ea640 1 -- 172.21.15.49:0/9928652 wait complete. 2024-03-14T22:05:28.754 INFO:tasks.ceph:Scrubbing osd.0 2024-03-14T22:05:28.754 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:28.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.877+0000 7fc588bea640 1 Processor -- start 2024-03-14T22:05:28.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.877+0000 7fc588bea640 1 -- start start 2024-03-14T22:05:28.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc588bea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bfc80 0x7fc58414ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:28.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc588bea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5840bb010 0x7fc5841555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:28.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc588bea640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58406ae40 con 0x7fc5840bc940 2024-03-14T22:05:28.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc588bea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5840ba750 con 0x7fc5840bb010 2024-03-14T22:05:28.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc588bea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5840bc590 con 0x7fc5840bfc80 2024-03-14T22:05:28.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc57bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc5840bc940 0x7fc5840b9c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43590/0 (socket says 172.21.15.49:43590) 2024-03-14T22:05:28.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc57bfff640 1 -- 172.21.15.49:0/704805098 learned_addr learned my addr 172.21.15.49:0/704805098 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc58259c640 1 --2- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bfc80 0x7fc58414ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc582d9d640 1 --2- 172.21.15.49:0/704805098 >> v2:172.21.15.97:3300/0 conn(0x7fc5840bb010 0x7fc5841555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.878+0000 7fc581d9b640 1 -- 172.21.15.49:0/704805098 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697070348 0 0) 0x7fc58406ae40 con 0x7fc5840bc940 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.879+0000 7fc581d9b640 1 -- 172.21.15.49:0/704805098 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc570003520 con 0x7fc5840bc940 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.879+0000 7fc582d9d640 1 -- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bfc80 msgr2=0x7fc58414ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.879+0000 7fc582d9d640 1 --2- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bfc80 0x7fc58414ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.879+0000 7fc582d9d640 1 -- 172.21.15.49:0/704805098 >> v1:172.21.15.49:6789/0 conn(0x7fc5840bc940 legacy=0x7fc5840b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.879+0000 7fc581d9b640 1 -- 172.21.15.49:0/704805098 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352926752 0 0) 0x7fc570003520 con 0x7fc5840bc940 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.879+0000 7fc582d9d640 1 -- 172.21.15.49:0/704805098 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5840bb4c0 con 0x7fc5840bb010 2024-03-14T22:05:28.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.879+0000 7fc58259c640 1 --2- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bfc80 0x7fc58414ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc582d9d640 1 --2- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bb010 0x7fc5841555c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009d90 tx=0x7fc57c02bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af967a5cc21f33f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc581d9b640 1 -- 172.21.15.49:0/704805098 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c001670 con 0x7fc5840bb010 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc581d9b640 1 -- 172.21.15.49:0/704805098 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c02c780 con 0x7fc5840bb010 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc581d9b640 1 -- 172.21.15.49:0/704805098 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c02cb10 con 0x7fc5840bb010 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 -- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bb010 msgr2=0x7fc5841555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 --2- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bb010 0x7fc5841555c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009d90 tx=0x7fc57c02bc50 comp rx=0 tx=0).stop 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 -- 172.21.15.49:0/704805098 shutdown_connections 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 --2- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bb010 0x7fc5841555c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 --2- 172.21.15.49:0/704805098 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bfc80 0x7fc58414ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 -- 172.21.15.49:0/704805098 >> 172.21.15.49:0/704805098 conn(0x7fc5840bcda0 msgr2=0x7fc5840bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 -- 172.21.15.49:0/704805098 shutdown_connections 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.880+0000 7fc588bea640 1 -- 172.21.15.49:0/704805098 wait complete. 2024-03-14T22:05:28.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 Processor -- start 2024-03-14T22:05:28.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 -- start start 2024-03-14T22:05:28.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5840bb010 0x7fc58409d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bc940 0x7fc5841e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bfc80 0x7fc5841e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58409ddb0 con 0x7fc5840bb010 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58409d480 con 0x7fc5840bfc80 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc588bea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58409f320 con 0x7fc5840bc940 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc58259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5840bb010 0x7fc58409d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc58259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5840bb010 0x7fc58409d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50180/0 (socket says 172.21.15.49:50180) 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.882+0000 7fc58259c640 1 -- 172.21.15.49:0/1870409135 learned_addr learned my addr 172.21.15.49:0/1870409135 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc57bfff640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bc940 0x7fc5841e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc582d9d640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bfc80 0x7fc5841e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc58259c640 1 -- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bc940 msgr2=0x7fc5841e37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc58259c640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5840bc940 0x7fc5841e37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc58259c640 1 -- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bfc80 msgr2=0x7fc5841e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc58259c640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5840bfc80 0x7fc5841e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc58259c640 1 -- 172.21.15.49:0/1870409135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574009400 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc58259c640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5840bb010 0x7fc58409d870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc57400e960 tx=0x7fc57400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57400c9a0 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57400cb00 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c009a70 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57400ce90 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.883+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58409f5a0 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.884+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc5840c0060 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc574013470 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc57affd640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc56406a7a0 0x7fc56406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc574089340 con 0x7fc5840bb010 2024-03-14T22:05:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc57bfff640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc56406a7a0 0x7fc56406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc57affd640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7fc564072ac0 0x7fc564074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 --> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc564075610 con 0x7fc564072ac0 2024-03-14T22:05:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc574089790 con 0x7fc5840bb010 2024-03-14T22:05:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.885+0000 7fc582d9d640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7fc564072ac0 0x7fc564074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.886+0000 7fc57bfff640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc56406a7a0 0x7fc56406cc60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc56c001b90 tx=0x7fc56c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.886+0000 7fc582d9d640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7fc564072ac0 0x7fc564074f40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:28.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.886+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== osd.0 v2:172.21.15.49:6816/37908 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc564075610 con 0x7fc564072ac0 2024-03-14T22:05:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.899+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 --> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc58406ab60 con 0x7fc564072ac0 2024-03-14T22:05:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.899+0000 7fc57affd640 1 -- 172.21.15.49:0/1870409135 <== osd.0 v2:172.21.15.49:6816/37908 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc57c02cd10 con 0x7fc564072ac0 2024-03-14T22:05:28.898 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:28.899 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:28.899 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7fc564072ac0 msgr2=0x7fc564074f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:6816/37908,v1:172.21.15.49:6817/37908] conn(0x7fc564072ac0 0x7fc564074f40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc56406a7a0 msgr2=0x7fc56406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fc56406a7a0 0x7fc56406cc60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc56c001b90 tx=0x7fc56c009040 comp rx=0 tx=0).stop 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5840bb010 msgr2=0x7fc58409d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 --2- 172.21.15.49:0/1870409135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5840bb010 0x7fc58409d870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc57400e960 tx=0x7fc57400ee20 comp rx=0 tx=0).stop 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc582d9d640 1 -- 172.21.15.49:0/1870409135 reap_dead start 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 shutdown_connections 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 >> 172.21.15.49:0/1870409135 conn(0x7fc5840bcda0 msgr2=0x7fc5840a0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 shutdown_connections 2024-03-14T22:05:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:28.900+0000 7fc588bea640 1 -- 172.21.15.49:0/1870409135 wait complete. 2024-03-14T22:05:28.910 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-14T22:05:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.018+0000 7f9950c69640 1 Processor -- start 2024-03-14T22:05:29.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.018+0000 7f9950c69640 1 -- start start 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f9950c69640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f994c0bb1c0 0x7f994c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f9950c69640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c0b3a20 con 0x7f994c0bad60 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f9950c69640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c06adf0 con 0x7f994c14b820 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f9950c69640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994c0bb5f0 con 0x7f994c0bb1c0 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f994ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f994c0bb1c0 0x7f994c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f9949d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f994c0bad60 0x7f994c153590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43604/0 (socket says 172.21.15.49:43604) 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f994ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f994c0bb1c0 0x7f994c156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50182/0 (socket says 172.21.15.49:50182) 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f9949d9b640 1 -- 172.21.15.49:0/2822584214 learned_addr learned my addr 172.21.15.49:0/2822584214 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f994ad9d640 1 -- 172.21.15.49:0/2822584214 >> v1:172.21.15.97:6789/0 conn(0x7f994c14b820 legacy=0x7f994c06a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f994ad9d640 1 -- 172.21.15.49:0/2822584214 >> v1:172.21.15.49:6789/0 conn(0x7f994c0bad60 legacy=0x7f994c153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f994ad9d640 1 -- 172.21.15.49:0/2822584214 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c0ba550 con 0x7f994c0bb1c0 2024-03-14T22:05:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.019+0000 7f994959a640 1 -- 172.21.15.49:0/2822584214 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952636704 0 0) 0x7f994c0b3a20 con 0x7f994c0bad60 2024-03-14T22:05:29.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.020+0000 7f994ad9d640 1 --2- 172.21.15.49:0/2822584214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c0bb1c0 0x7f994c156cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d90 tx=0x7f993c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aa8298ac207465 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.020+0000 7f994959a640 1 -- 172.21.15.49:0/2822584214 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c02cc60 con 0x7f994c0bb1c0 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.020+0000 7f994959a640 1 -- 172.21.15.49:0/2822584214 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c02cdc0 con 0x7f994c0bb1c0 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.020+0000 7f994959a640 1 -- 172.21.15.49:0/2822584214 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c034660 con 0x7f994c0bb1c0 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.021+0000 7f9950c69640 1 -- 172.21.15.49:0/2822584214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c0bb1c0 msgr2=0x7f994c156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.021+0000 7f9950c69640 1 --2- 172.21.15.49:0/2822584214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c0bb1c0 0x7f994c156cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d90 tx=0x7f993c02beb0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.021+0000 7f9950c69640 1 -- 172.21.15.49:0/2822584214 shutdown_connections 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.021+0000 7f9950c69640 1 --2- 172.21.15.49:0/2822584214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c0bb1c0 0x7f994c156cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.021+0000 7f9950c69640 1 -- 172.21.15.49:0/2822584214 >> 172.21.15.49:0/2822584214 conn(0x7f994c0bcd80 msgr2=0x7f994c0bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.021+0000 7f9950c69640 1 -- 172.21.15.49:0/2822584214 shutdown_connections 2024-03-14T22:05:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.021+0000 7f9950c69640 1 -- 172.21.15.49:0/2822584214 wait complete. 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.022+0000 7f9950c69640 1 Processor -- start 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.022+0000 7f9950c69640 1 -- start start 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9950c69640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f994c0bad60 0x7f994c09c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9950c69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f994c0bb1c0 0x7f994c09b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9950c69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c14b820 0x7f994c09b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9950c69640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1e3160 con 0x7f994c14b820 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9950c69640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c09cda0 con 0x7f994c0bb1c0 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9950c69640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994c09bc20 con 0x7f994c0bad60 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c14b820 0x7f994c09b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c14b820 0x7f994c09b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50184/0 (socket says 172.21.15.49:50184) 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 -- 172.21.15.49:0/1902737941 learned_addr learned my addr 172.21.15.49:0/1902737941 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9949d9b640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f994c0bb1c0 0x7f994c09b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994a59c640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f994c0bad60 0x7f994c09c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 -- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f994c0bad60 msgr2=0x7f994c09c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f994c0bad60 0x7f994c09c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 -- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f994c0bb1c0 msgr2=0x7f994c09b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f994c0bb1c0 0x7f994c09b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 -- 172.21.15.49:0/1902737941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c009a70 con 0x7f994c14b820 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9949d9b640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f994c0bb1c0 0x7f994c09b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f994ad9d640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c14b820 0x7f994c09b620 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d90 tx=0x7f993c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f993affd640 1 -- 172.21.15.49:0/1902737941 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c03e610 con 0x7f994c14b820 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f993affd640 1 -- 172.21.15.49:0/1902737941 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c03e790 con 0x7f994c14b820 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c0b6970 con 0x7f994c14b820 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.023+0000 7f993affd640 1 -- 172.21.15.49:0/1902737941 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c03eb40 con 0x7f994c14b820 2024-03-14T22:05:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.024+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c149090 con 0x7f994c14b820 2024-03-14T22:05:29.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.024+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c1e32d0 con 0x7f994c14b820 2024-03-14T22:05:29.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.025+0000 7f993affd640 1 -- 172.21.15.49:0/1902737941 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f993c03b050 con 0x7f994c14b820 2024-03-14T22:05:29.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.025+0000 7f993affd640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f992c06a7d0 0x7f992c06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.025+0000 7f993affd640 1 -- 172.21.15.49:0/1902737941 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f993c0ab480 con 0x7f994c14b820 2024-03-14T22:05:29.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.026+0000 7f994a59c640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f992c06a7d0 0x7f992c06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.026+0000 7f994a59c640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f992c06a7d0 0x7f992c06cc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9944006fd0 tx=0x7f9944008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.030+0000 7f993affd640 1 -- 172.21.15.49:0/1902737941 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f993c0768b0 con 0x7f994c14b820 2024-03-14T22:05:29.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.208+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f994c0bc590 con 0x7f992c06a7d0 2024-03-14T22:05:29.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.209+0000 7f993affd640 1 -- 172.21.15.49:0/1902737941 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f994400be30 con 0x7f992c06a7d0 2024-03-14T22:05:29.208 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 0 to deep-scrub 2024-03-14T22:05:29.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f992c06a7d0 msgr2=0x7f992c06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f992c06a7d0 0x7f992c06cc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9944006fd0 tx=0x7f9944008040 comp rx=0 tx=0).stop 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c14b820 msgr2=0x7f994c09b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c14b820 0x7f994c09b620 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f993c009d90 tx=0x7f993c004a40 comp rx=0 tx=0).stop 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 shutdown_connections 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f992c06a7d0 0x7f992c06cc90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f994c14b820 0x7f994c09b620 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f994c0bb1c0 0x7f994c09b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 --2- 172.21.15.49:0/1902737941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f994c0bad60 0x7f994c09c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 >> 172.21.15.49:0/1902737941 conn(0x7f994c0bcd80 msgr2=0x7f994c0a0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 shutdown_connections 2024-03-14T22:05:29.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.211+0000 7f9950c69640 1 -- 172.21.15.49:0/1902737941 wait complete. 2024-03-14T22:05:29.221 INFO:tasks.ceph:Scrubbing osd.1 2024-03-14T22:05:29.222 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:29.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.332+0000 7fa19cffb640 1 Processor -- start 2024-03-14T22:05:29.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.332+0000 7fa19cffb640 1 -- start start 2024-03-14T22:05:29.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19cffb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 0x7fa19814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19cffb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19806ae40 con 0x7fa1980bc940 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19cffb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980bc590 con 0x7fa1980bb010 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19cffb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980ba750 con 0x7fa1980bfc80 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa195d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa1980bc940 0x7fa1980b9c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43616/0 (socket says 172.21.15.49:43616) 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa195d9b640 1 -- 172.21.15.49:0/3948662092 learned_addr learned my addr 172.21.15.49:0/3948662092 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19659c640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 0x7fa19814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060621541 0 0) 0x7fa19806ae40 con 0x7fa1980bc940 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa180003520 con 0x7fa1980bc940 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.333+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3739088164 0 0) 0x7fa180003520 con 0x7fa1980bc940 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 msgr2=0x7fa19814ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 0x7fa19814ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 >> v1:172.21.15.97:6789/0 conn(0x7fa1980bb010 legacy=0x7fa1981555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980bb4c0 con 0x7fa1980bc940 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7fa18c001670 con 0x7fa1980bc940 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 >> v1:172.21.15.49:6789/0 conn(0x7fa1980bc940 legacy=0x7fa1980b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180003a80 0x7fa180003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 0x7fa18000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980ba750 con 0x7fa180003a80 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800054e0 con 0x7fa1800047c0 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18000bec0 con 0x7fa1980bfc80 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19659c640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180003a80 0x7fa180003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa195d9b640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa196d9d640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 0x7fa18000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19659c640 1 -- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 msgr2=0x7fa18000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19659c640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1980bfc80 0x7fa18000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19659c640 1 -- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 msgr2=0x7fa180004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19659c640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.334+0000 7fa19659c640 1 -- 172.21.15.49:0/3948662092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180003520 con 0x7fa180003a80 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19659c640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180003a80 0x7fa180003e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa190009ac0 tx=0x7fa1800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28380725cbdaede4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19002c6b0 con 0x7fa180003a80 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19002c810 con 0x7fa180003a80 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19559a640 1 -- 172.21.15.49:0/3948662092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19002cba0 con 0x7fa180003a80 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19cffb640 1 -- 172.21.15.49:0/3948662092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180003a80 msgr2=0x7fa180003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19cffb640 1 --2- 172.21.15.49:0/3948662092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa180003a80 0x7fa180003e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa190009ac0 tx=0x7fa1800057f0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa196d9d640 1 -- 172.21.15.49:0/3948662092 reap_dead start 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19cffb640 1 -- 172.21.15.49:0/3948662092 shutdown_connections 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.335+0000 7fa19cffb640 1 -- 172.21.15.49:0/3948662092 >> 172.21.15.49:0/3948662092 conn(0x7fa1980bcda0 msgr2=0x7fa1980bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.336+0000 7fa19cffb640 1 -- 172.21.15.49:0/3948662092 shutdown_connections 2024-03-14T22:05:29.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.336+0000 7fa19cffb640 1 -- 172.21.15.49:0/3948662092 wait complete. 2024-03-14T22:05:29.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 Processor -- start 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 -- start start 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa180003a80 0x7fa1981e0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa19809f730 0x7fa198099cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19809af00 0x7fa19814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19814efb0 con 0x7fa19809af00 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981dfd00 con 0x7fa180003a80 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa19cffb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19809a2e0 con 0x7fa19809f730 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa196d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19809af00 0x7fa19814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa196d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19809af00 0x7fa19814ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50216/0 (socket says 172.21.15.49:50216) 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.337+0000 7fa196d9d640 1 -- 172.21.15.49:0/1072938929 learned_addr learned my addr 172.21.15.49:0/1072938929 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa19659c640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa180003a80 0x7fa1981e0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa195d9b640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa19809f730 0x7fa198099cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa196d9d640 1 -- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa19809f730 msgr2=0x7fa198099cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa196d9d640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa19809f730 0x7fa198099cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa196d9d640 1 -- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa180003a80 msgr2=0x7fa1981e0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa196d9d640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa180003a80 0x7fa1981e0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa196d9d640 1 -- 172.21.15.49:0/1072938929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1981df6b0 con 0x7fa19809af00 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa19659c640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa180003a80 0x7fa1981e0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa195d9b640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa19809f730 0x7fa198099cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa196d9d640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19809af00 0x7fa19814ea70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa198066650 tx=0x7fa18800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18800ee50 con 0x7fa19809af00 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188003b80 con 0x7fa19809af00 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1981df990 con 0x7fa19809af00 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880050d0 con 0x7fa19809af00 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.338+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa198146cb0 con 0x7fa19809af00 2024-03-14T22:05:29.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.339+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1980c0060 con 0x7fa19809af00 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.339+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa188005230 con 0x7fa19809af00 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa186ffd640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa17806a7d0 0x7fa17806cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa188088ce0 con 0x7fa19809af00 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa186ffd640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7fa178072af0 0x7fa178074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 --> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa178075640 con 0x7fa178072af0 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa18808b370 con 0x7fa19809af00 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa19659c640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa17806a7d0 0x7fa17806cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa195d9b640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7fa178072af0 0x7fa178074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.340+0000 7fa195d9b640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7fa178072af0 0x7fa178074f70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.341+0000 7fa19659c640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa17806a7d0 0x7fa17806cc90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa190036000 tx=0x7fa190030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.341+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== osd.1 v2:172.21.15.49:6800/37907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa178075640 con 0x7fa178072af0 2024-03-14T22:05:29.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 --> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa1980b8870 con 0x7fa178072af0 2024-03-14T22:05:29.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa186ffd640 1 -- 172.21.15.49:0/1072938929 <== osd.1 v2:172.21.15.49:6800/37907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa1980b8870 con 0x7fa178072af0 2024-03-14T22:05:29.353 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:29.353 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:29.353 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7fa178072af0 msgr2=0x7fa178074f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:6800/37907,v1:172.21.15.49:6801/37907] conn(0x7fa178072af0 0x7fa178074f70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa17806a7d0 msgr2=0x7fa17806cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fa17806a7d0 0x7fa17806cc90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa190036000 tx=0x7fa190030000 comp rx=0 tx=0).stop 2024-03-14T22:05:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19809af00 msgr2=0x7fa19814ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 --2- 172.21.15.49:0/1072938929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa19809af00 0x7fa19814ea70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa198066650 tx=0x7fa18800ec00 comp rx=0 tx=0).stop 2024-03-14T22:05:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa196d9d640 1 -- 172.21.15.49:0/1072938929 reap_dead start 2024-03-14T22:05:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 shutdown_connections 2024-03-14T22:05:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 >> 172.21.15.49:0/1072938929 conn(0x7fa1980bcda0 msgr2=0x7fa1980a1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 shutdown_connections 2024-03-14T22:05:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.354+0000 7fa19cffb640 1 -- 172.21.15.49:0/1072938929 wait complete. 2024-03-14T22:05:29.364 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-14T22:05:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.472+0000 7f77d5769640 1 Processor -- start 2024-03-14T22:05:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.472+0000 7f77d5769640 1 -- start start 2024-03-14T22:05:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.472+0000 7f77d5769640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77d014b820 0x7f77d006a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77d5769640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d00bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77d5769640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00b3a20 con 0x7f77d014e270 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77d5769640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d00ba550 con 0x7f77d014b820 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77d5769640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00bfbe0 con 0x7f77d00bad60 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77ceffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77d014b820 0x7f77d006a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77c67fc640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d00bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77cf7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77d014e270 0x7f77d014e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:57170/0 (socket says 172.21.15.49:57170) 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77cf7fe640 1 -- 172.21.15.49:0/548264831 learned_addr learned my addr 172.21.15.49:0/548264831 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77ceffd640 1 -- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 msgr2=0x7f77d00bb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77ceffd640 1 --2- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d00bb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77ceffd640 1 -- 172.21.15.49:0/548264831 >> v1:172.21.15.97:6789/0 conn(0x7f77d014e270 legacy=0x7f77d014e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77ceffd640 1 -- 172.21.15.49:0/548264831 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00bfec0 con 0x7f77d014b820 2024-03-14T22:05:29.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.473+0000 7f77c67fc640 1 --2- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d00bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T22:05:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.474+0000 7f77ceffd640 1 --2- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014b820 0x7f77d006a6e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77c00097a0 tx=0x7f77c002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854ebc74d3346b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.474+0000 7f77ce7fc640 1 -- 172.21.15.49:0/548264831 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0031830 con 0x7f77d014b820 2024-03-14T22:05:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.474+0000 7f77ce7fc640 1 -- 172.21.15.49:0/548264831 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0031990 con 0x7f77d014b820 2024-03-14T22:05:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.474+0000 7f77ce7fc640 1 -- 172.21.15.49:0/548264831 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0031d20 con 0x7f77d014b820 2024-03-14T22:05:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 -- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014b820 msgr2=0x7f77d006a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 --2- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014b820 0x7f77d006a6e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77c00097a0 tx=0x7f77c002b8d0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 -- 172.21.15.49:0/548264831 shutdown_connections 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 --2- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d00bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 --2- 172.21.15.49:0/548264831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014b820 0x7f77d006a6e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 -- 172.21.15.49:0/548264831 >> 172.21.15.49:0/548264831 conn(0x7f77d00bcd80 msgr2=0x7f77d00bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 -- 172.21.15.49:0/548264831 shutdown_connections 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.475+0000 7f77d5769640 1 -- 172.21.15.49:0/548264831 wait complete. 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.476+0000 7f77d5769640 1 Processor -- start 2024-03-14T22:05:29.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.476+0000 7f77d5769640 1 -- start start 2024-03-14T22:05:29.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77d5769640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d01d7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77d5769640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d014b820 0x7f77d01d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77d5769640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014e270 0x7f77d01d6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77d5769640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01d7680 con 0x7f77d014b820 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77d5769640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01d5840 con 0x7f77d014e270 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77d5769640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01d4f10 con 0x7f77d00bad60 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d014b820 0x7f77d01d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d014b820 0x7f77d01d5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50224/0 (socket says 172.21.15.49:50224) 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 -- 172.21.15.49:0/3216788413 learned_addr learned my addr 172.21.15.49:0/3216788413 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77cf7fe640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014e270 0x7f77d01d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77ceffd640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d01d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 -- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 msgr2=0x7f77d01d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d01d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 -- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014e270 msgr2=0x7f77d01d6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014e270 0x7f77d01d6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 -- 172.21.15.49:0/3216788413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8009400 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c67fc640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d014b820 0x7f77d01d5f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f77b800ebb0 tx=0x7f77b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.477+0000 7f77c7fff640 1 -- 172.21.15.49:0/3216788413 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800eeb0 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.478+0000 7f77c7fff640 1 -- 172.21.15.49:0/3216788413 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b800cc30 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.478+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c0009400 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.478+0000 7f77c7fff640 1 -- 172.21.15.49:0/3216788413 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8013470 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.478+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d01d54d0 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.479+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d01e33e0 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.479+0000 7f77c7fff640 1 -- 172.21.15.49:0/3216788413 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f77b80135d0 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.479+0000 7f77c7fff640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f77b006a7d0 0x7f77b006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.479+0000 7f77c7fff640 1 -- 172.21.15.49:0/3216788413 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f77b8089070 con 0x7f77d014b820 2024-03-14T22:05:29.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.480+0000 7f77ceffd640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f77b006a7d0 0x7f77b006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.480+0000 7f77ceffd640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f77b006a7d0 0x7f77b006cc90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f77c0036000 tx=0x7f77c003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.484+0000 7f77c7fff640 1 -- 172.21.15.49:0/3216788413 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f77b80544a0 con 0x7f77d014b820 2024-03-14T22:05:29.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.668+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f77d014ad70 con 0x7f77b006a7d0 2024-03-14T22:05:29.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.668+0000 7f77c7fff640 1 -- 172.21.15.49:0/3216788413 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f77c003a420 con 0x7f77b006a7d0 2024-03-14T22:05:29.667 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 1 to deep-scrub 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f77b006a7d0 msgr2=0x7f77b006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f77b006a7d0 0x7f77b006cc90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f77c0036000 tx=0x7f77c003c040 comp rx=0 tx=0).stop 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d014b820 msgr2=0x7f77d01d5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d014b820 0x7f77d01d5f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f77b800ebb0 tx=0x7f77b800c520 comp rx=0 tx=0).stop 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 shutdown_connections 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f77b006a7d0 0x7f77b006cc90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77d014e270 0x7f77d01d6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77d014b820 0x7f77d01d5f20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 --2- 172.21.15.49:0/3216788413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77d00bad60 0x7f77d01d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.670+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 >> 172.21.15.49:0/3216788413 conn(0x7f77d00bcd80 msgr2=0x7f77d00bd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.671+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 shutdown_connections 2024-03-14T22:05:29.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.671+0000 7f77d5769640 1 -- 172.21.15.49:0/3216788413 wait complete. 2024-03-14T22:05:29.680 INFO:tasks.ceph:Scrubbing osd.2 2024-03-14T22:05:29.681 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:29.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.791+0000 7f3749f5c640 1 Processor -- start 2024-03-14T22:05:29.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.791+0000 7f3749f5c640 1 -- start start 2024-03-14T22:05:29.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3749f5c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f37440bc940 0x7f374414ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3749f5c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374406ae40 con 0x7f37440bfc80 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3749f5c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440bc590 con 0x7f37440bb010 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3749f5c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37440ba750 con 0x7f37440bc940 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3742ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f37440bc940 0x7f374414ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3742ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f37440bc940 0x7f374414ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50230/0 (socket says 172.21.15.49:50230) 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f37437fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f37440bfc80 0x7f37440b9c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43626/0 (socket says 172.21.15.49:43626) 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3742ffd640 1 -- 172.21.15.49:0/3599034094 learned_addr learned my addr 172.21.15.49:0/3599034094 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3742ffd640 1 -- 172.21.15.49:0/3599034094 >> v1:172.21.15.97:6789/0 conn(0x7f37440bb010 legacy=0x7f37441555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3742ffd640 1 -- 172.21.15.49:0/3599034094 >> v1:172.21.15.49:6789/0 conn(0x7f37440bfc80 legacy=0x7f37440b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.792+0000 7f3742ffd640 1 -- 172.21.15.49:0/3599034094 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374406a880 con 0x7f37440bc940 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.793+0000 7f3742ffd640 1 --2- 172.21.15.49:0/3599034094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bc940 0x7f374414ad90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3738009870 tx=0x7f373802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403f3a936d416ffd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.793+0000 7f37427fc640 1 -- 172.21.15.49:0/3599034094 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380317e0 con 0x7f37440bc940 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.793+0000 7f37427fc640 1 -- 172.21.15.49:0/3599034094 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3738031940 con 0x7f37440bc940 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.793+0000 7f37427fc640 1 -- 172.21.15.49:0/3599034094 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738031cd0 con 0x7f37440bc940 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.794+0000 7f3749f5c640 1 -- 172.21.15.49:0/3599034094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bc940 msgr2=0x7f374414ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.794+0000 7f3749f5c640 1 --2- 172.21.15.49:0/3599034094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bc940 0x7f374414ad90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3738009870 tx=0x7f373802b880 comp rx=0 tx=0).stop 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.794+0000 7f3749f5c640 1 -- 172.21.15.49:0/3599034094 shutdown_connections 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.794+0000 7f3749f5c640 1 --2- 172.21.15.49:0/3599034094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bc940 0x7f374414ad90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.794+0000 7f3749f5c640 1 -- 172.21.15.49:0/3599034094 >> 172.21.15.49:0/3599034094 conn(0x7f37440bcda0 msgr2=0x7f37440bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.794+0000 7f3749f5c640 1 -- 172.21.15.49:0/3599034094 shutdown_connections 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.794+0000 7f3749f5c640 1 -- 172.21.15.49:0/3599034094 wait complete. 2024-03-14T22:05:29.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.795+0000 7f3749f5c640 1 Processor -- start 2024-03-14T22:05:29.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.795+0000 7f3749f5c640 1 -- start start 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bb010 0x7f37440a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440bc940 0x7f3744144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37440bfc80 0x7f37441e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374414e2e0 con 0x7f37440bb010 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440a6040 con 0x7f37440bc940 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374414d230 con 0x7f37440bfc80 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bb010 0x7f37440a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bb010 0x7f37440a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50238/0 (socket says 172.21.15.49:50238) 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 -- 172.21.15.49:0/4044181352 learned_addr learned my addr 172.21.15.49:0/4044181352 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3742ffd640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440bc940 0x7f3744144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3743fff640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37440bfc80 0x7f37441e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 -- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37440bfc80 msgr2=0x7f37441e00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37440bfc80 0x7f37441e00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 -- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440bc940 msgr2=0x7f3744144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440bc940 0x7f3744144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 -- 172.21.15.49:0/4044181352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380094d0 con 0x7f37440bb010 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f37437fe640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bb010 0x7f37440a5b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f373400c3e0 tx=0x7f373400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340074e0 con 0x7f37440bb010 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734007640 con 0x7f37440bb010 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374414ec60 con 0x7f37440bb010 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340049e0 con 0x7f37440bb010 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.796+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374414e8f0 con 0x7f37440bb010 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.797+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f374414f0f0 con 0x7f37440bb010 2024-03-14T22:05:29.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.798+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3734004b40 con 0x7f37440bb010 2024-03-14T22:05:29.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.798+0000 7f372bfff640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f373006a7a0 0x7f373006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.798+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3734088780 con 0x7f37440bb010 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.798+0000 7f3742ffd640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f373006a7a0 0x7f373006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.798+0000 7f372bfff640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7f3730072ac0 0x7f3730074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.798+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 --> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3730075610 con 0x7f3730072ac0 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.798+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f373408ae40 con 0x7f37440bb010 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.799+0000 7f3743fff640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7f3730072ac0 0x7f3730074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.799+0000 7f3743fff640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7f3730072ac0 0x7f3730074f40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.799+0000 7f3742ffd640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f373006a7a0 0x7f373006cc60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3738036000 tx=0x7f373803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.799+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== osd.2 v2:172.21.15.49:6824/37918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f3730075610 con 0x7f3730072ac0 2024-03-14T22:05:29.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.812+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 --> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f37440c0060 con 0x7f3730072ac0 2024-03-14T22:05:29.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.812+0000 7f372bfff640 1 -- 172.21.15.49:0/4044181352 <== osd.2 v2:172.21.15.49:6824/37918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f373c001980 con 0x7f3730072ac0 2024-03-14T22:05:29.811 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:29.812 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:29.812 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.812+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7f3730072ac0 msgr2=0x7f3730074f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:6824/37918,v1:172.21.15.49:6825/37918] conn(0x7f3730072ac0 0x7f3730074f40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f373006a7a0 msgr2=0x7f373006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f373006a7a0 0x7f373006cc60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3738036000 tx=0x7f373803d040 comp rx=0 tx=0).stop 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bb010 msgr2=0x7f37440a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 --2- 172.21.15.49:0/4044181352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37440bb010 0x7f37440a5b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f373400c3e0 tx=0x7f373400c8a0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3743fff640 1 -- 172.21.15.49:0/4044181352 reap_dead start 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 shutdown_connections 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 >> 172.21.15.49:0/4044181352 conn(0x7f37440bcda0 msgr2=0x7f37440a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 shutdown_connections 2024-03-14T22:05:29.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.813+0000 7f3749f5c640 1 -- 172.21.15.49:0/4044181352 wait complete. 2024-03-14T22:05:29.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-14T22:05:29.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.929+0000 7f877093e640 1 Processor -- start 2024-03-14T22:05:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.929+0000 7f877093e640 1 -- start start 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f877093e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f876c142cf0 0x7f876c0ba600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f877093e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f876c0b3260 0x7f876c06dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f877093e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0bab40 con 0x7f876c06e6a0 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f877093e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c0b3740 con 0x7f876c142cf0 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f877093e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c0b4ec0 con 0x7f876c0b3260 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f876a59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f876c142cf0 0x7f876c0ba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f876ad9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f876c06e6a0 0x7f876c14bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43628/0 (socket says 172.21.15.49:43628) 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.930+0000 7f876ad9d640 1 -- 172.21.15.49:0/2743542732 learned_addr learned my addr 172.21.15.49:0/2743542732 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.931+0000 7f8769d9b640 1 --2- 172.21.15.49:0/2743542732 >> v2:172.21.15.97:3300/0 conn(0x7f876c0b3260 0x7f876c06dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.931+0000 7f876a59c640 1 -- 172.21.15.49:0/2743542732 >> v2:172.21.15.97:3300/0 conn(0x7f876c0b3260 msgr2=0x7f876c06dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.931+0000 7f876a59c640 1 --2- 172.21.15.49:0/2743542732 >> v2:172.21.15.97:3300/0 conn(0x7f876c0b3260 0x7f876c06dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.931+0000 7f876959a640 1 -- 172.21.15.49:0/2743542732 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770062047 0 0) 0x7f876c0bab40 con 0x7f876c06e6a0 2024-03-14T22:05:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.931+0000 7f876a59c640 1 -- 172.21.15.49:0/2743542732 >> v1:172.21.15.49:6789/0 conn(0x7f876c06e6a0 legacy=0x7f876c14bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.931+0000 7f876a59c640 1 -- 172.21.15.49:0/2743542732 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0b51a0 con 0x7f876c142cf0 2024-03-14T22:05:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.931+0000 7f876a59c640 1 --2- 172.21.15.49:0/2743542732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c142cf0 0x7f876c0ba600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f876c0b9ee0 tx=0x7f875402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10123b75f9b3a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f876959a640 1 -- 172.21.15.49:0/2743542732 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031830 con 0x7f876c142cf0 2024-03-14T22:05:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f876959a640 1 -- 172.21.15.49:0/2743542732 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754031990 con 0x7f876c142cf0 2024-03-14T22:05:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f876959a640 1 -- 172.21.15.49:0/2743542732 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031d20 con 0x7f876c142cf0 2024-03-14T22:05:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 -- 172.21.15.49:0/2743542732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c142cf0 msgr2=0x7f876c0ba600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 --2- 172.21.15.49:0/2743542732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c142cf0 0x7f876c0ba600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f876c0b9ee0 tx=0x7f875402b8d0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 -- 172.21.15.49:0/2743542732 shutdown_connections 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 --2- 172.21.15.49:0/2743542732 >> v2:172.21.15.97:3300/0 conn(0x7f876c0b3260 0x7f876c06dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 --2- 172.21.15.49:0/2743542732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c142cf0 0x7f876c0ba600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 -- 172.21.15.49:0/2743542732 >> 172.21.15.49:0/2743542732 conn(0x7f876c0bad40 msgr2=0x7f876c0bb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 -- 172.21.15.49:0/2743542732 shutdown_connections 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.932+0000 7f877093e640 1 -- 172.21.15.49:0/2743542732 wait complete. 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.933+0000 7f877093e640 1 Processor -- start 2024-03-14T22:05:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.933+0000 7f877093e640 1 -- start start 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f877093e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f876c06e6a0 0x7f876c06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f877093e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c0b3260 0x7f876c14c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f877093e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f876c142cf0 0x7f876c14a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f877093e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c14caf0 con 0x7f876c0b3260 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f877093e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c149cf0 con 0x7f876c142cf0 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f877093e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c06b090 con 0x7f876c06e6a0 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c0b3260 0x7f876c14c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c0b3260 0x7f876c14c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50260/0 (socket says 172.21.15.49:50260) 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 -- 172.21.15.49:0/3558473081 learned_addr learned my addr 172.21.15.49:0/3558473081 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f876ad9d640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f876c142cf0 0x7f876c14a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f876a59c640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f876c06e6a0 0x7f876c06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 -- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f876c06e6a0 msgr2=0x7f876c06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f876c06e6a0 0x7f876c06c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 -- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f876c142cf0 msgr2=0x7f876c14a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f876c142cf0 0x7f876c14a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.934+0000 7f8769d9b640 1 -- 172.21.15.49:0/3558473081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8754009400 con 0x7f876c0b3260 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.935+0000 7f876ad9d640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f876c142cf0 0x7f876c14a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.935+0000 7f8769d9b640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c0b3260 0x7f876c14c5b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c820 tx=0x7f875c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.935+0000 7f8762ffd640 1 -- 172.21.15.49:0/3558473081 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c004220 con 0x7f876c0b3260 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.935+0000 7f8762ffd640 1 -- 172.21.15.49:0/3558473081 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c004380 con 0x7f876c0b3260 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.935+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c14afe0 con 0x7f876c0b3260 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.935+0000 7f8762ffd640 1 -- 172.21.15.49:0/3558473081 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0079f0 con 0x7f876c0b3260 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.935+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c0b8410 con 0x7f876c0b3260 2024-03-14T22:05:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.936+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c0b43f0 con 0x7f876c0b3260 2024-03-14T22:05:29.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.936+0000 7f8762ffd640 1 -- 172.21.15.49:0/3558473081 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f875c007b50 con 0x7f876c0b3260 2024-03-14T22:05:29.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.937+0000 7f8762ffd640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f874c06a7a0 0x7f874c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:29.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.937+0000 7f8762ffd640 1 -- 172.21.15.49:0/3558473081 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f875c088b80 con 0x7f876c0b3260 2024-03-14T22:05:29.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.937+0000 7f876a59c640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f874c06a7a0 0x7f874c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:29.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.937+0000 7f876a59c640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f874c06a7a0 0x7f874c06cc60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8754036000 tx=0x7f875403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:29.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:29.941+0000 7f8762ffd640 1 -- 172.21.15.49:0/3558473081 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f875c053f30 con 0x7f876c0b3260 2024-03-14T22:05:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.119+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f876c06eb50 con 0x7f874c06a7a0 2024-03-14T22:05:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.120+0000 7f8762ffd640 1 -- 172.21.15.49:0/3558473081 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f875403a420 con 0x7f874c06a7a0 2024-03-14T22:05:30.119 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 2 to deep-scrub 2024-03-14T22:05:30.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f874c06a7a0 msgr2=0x7f874c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f874c06a7a0 0x7f874c06cc60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8754036000 tx=0x7f875403c040 comp rx=0 tx=0).stop 2024-03-14T22:05:30.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c0b3260 msgr2=0x7f876c14c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c0b3260 0x7f876c14c5b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c820 tx=0x7f875c00cce0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 shutdown_connections 2024-03-14T22:05:30.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f874c06a7a0 0x7f874c06cc60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f876c142cf0 0x7f876c14a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f876c0b3260 0x7f876c14c5b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 --2- 172.21.15.49:0/3558473081 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f876c06e6a0 0x7f876c06c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 >> 172.21.15.49:0/3558473081 conn(0x7f876c0bad40 msgr2=0x7f876c0b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 shutdown_connections 2024-03-14T22:05:30.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.122+0000 7f877093e640 1 -- 172.21.15.49:0/3558473081 wait complete. 2024-03-14T22:05:30.135 INFO:tasks.ceph:Scrubbing osd.3 2024-03-14T22:05:30.135 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.251+0000 7f9a836df640 1 Processor -- start 2024-03-14T22:05:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.251+0000 7f9a836df640 1 -- start start 2024-03-14T22:05:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a836df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 0x7f9a7c14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a836df640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c06ae40 con 0x7f9a7c0bb010 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a836df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c0bc590 con 0x7f9a7c0bfc80 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a836df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0ba750 con 0x7f9a7c0bc940 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a81c7c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9a7c0bb010 0x7f9a7c1555c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43630/0 (socket says 172.21.15.49:43630) 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a81c7c640 1 -- 172.21.15.49:0/3980910946 learned_addr learned my addr 172.21.15.49:0/3980910946 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a80c7a640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 0x7f9a7c14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067725332 0 0) 0x7f9a7c06ae40 con 0x7f9a7c0bb010 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a68003520 con 0x7f9a7c0bb010 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4079804208 0 0) 0x7f9a68003520 con 0x7f9a7c0bb010 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 msgr2=0x7f9a7c14ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 0x7f9a7c14ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 >> v1:172.21.15.97:6789/0 conn(0x7f9a7c0bfc80 legacy=0x7f9a7c0b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0bb4c0 con 0x7f9a7c0bb010 2024-03-14T22:05:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f9a740019f0 con 0x7f9a7c0bb010 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 >> v1:172.21.15.49:6789/0 conn(0x7f9a7c0bb010 legacy=0x7f9a7c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a68003a80 0x7f9a68003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 0x7f9a680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68004d70 0x7f9a680051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0ba750 con 0x7f9a68003a80 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6800bc50 con 0x7f9a68004d70 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.252+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6800be10 con 0x7f9a7c0bc940 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a8147b640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a68003a80 0x7f9a68003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a80c7a640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 0x7f9a680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a81c7c640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68004d70 0x7f9a680051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a8147b640 1 -- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 msgr2=0x7f9a680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a8147b640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a7c0bc940 0x7f9a680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a8147b640 1 -- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68004d70 msgr2=0x7f9a680051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a8147b640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68004d70 0x7f9a680051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.253+0000 7f9a8147b640 1 -- 172.21.15.49:0/3980910946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a68003520 con 0x7f9a68003a80 2024-03-14T22:05:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a8147b640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a68003a80 0x7f9a68003ea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9a78001750 tx=0x7f9a680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c92dc09493715d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780052d0 con 0x7f9a68003a80 2024-03-14T22:05:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7802e050 con 0x7f9a68003a80 2024-03-14T22:05:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a73fff640 1 -- 172.21.15.49:0/3980910946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780035f0 con 0x7f9a68003a80 2024-03-14T22:05:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a836df640 1 -- 172.21.15.49:0/3980910946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a68003a80 msgr2=0x7f9a68003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a836df640 1 --2- 172.21.15.49:0/3980910946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a68003a80 0x7f9a68003ea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9a78001750 tx=0x7f9a680057f0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a81c7c640 1 -- 172.21.15.49:0/3980910946 reap_dead start 2024-03-14T22:05:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a836df640 1 -- 172.21.15.49:0/3980910946 shutdown_connections 2024-03-14T22:05:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a836df640 1 -- 172.21.15.49:0/3980910946 >> 172.21.15.49:0/3980910946 conn(0x7f9a7c0bcda0 msgr2=0x7f9a7c0bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a836df640 1 -- 172.21.15.49:0/3980910946 shutdown_connections 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.254+0000 7f9a836df640 1 -- 172.21.15.49:0/3980910946 wait complete. 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.255+0000 7f9a836df640 1 Processor -- start 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.255+0000 7f9a836df640 1 -- start start 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a836df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a68003a80 0x7f9a7c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a836df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c0a5cf0 0x7f9a7c0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a836df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1dfc90 0x7f9a7c146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a836df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c14ee50 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a836df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0b5610 con 0x7f9a7c0a5cf0 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a836df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0b46d0 con 0x7f9a68003a80 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1dfc90 0x7f9a7c146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1dfc90 0x7f9a7c146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50264/0 (socket says 172.21.15.49:50264) 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 -- 172.21.15.49:0/2187750447 learned_addr learned my addr 172.21.15.49:0/2187750447 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a80c7a640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c0a5cf0 0x7f9a7c0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a8147b640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a68003a80 0x7f9a7c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 -- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a68003a80 msgr2=0x7f9a7c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9a68003a80 0x7f9a7c14e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 -- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c0a5cf0 msgr2=0x7f9a7c0b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a7c0a5cf0 0x7f9a7c0b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.256+0000 7f9a81c7c640 1 -- 172.21.15.49:0/2187750447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0a5210 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.257+0000 7f9a81c7c640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1dfc90 0x7f9a7c146eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c14f350 tx=0x7f9a74007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.257+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400aab0 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.257+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7400ac50 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.257+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c0a54f0 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.257+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a740133f0 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.257+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c0b5820 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.258+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a7c0c0060 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.258+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9a74013550 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.258+0000 7f9a71ffb640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9a6006a7a0 0x7f9a6006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9a740893f0 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a8147b640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9a6006a7a0 0x7f9a6006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a71ffb640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7f9a60072ac0 0x7f9a60074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 --> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a60075610 con 0x7f9a60072ac0 2024-03-14T22:05:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a7408bab0 con 0x7f9a7c1dfc90 2024-03-14T22:05:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a80c7a640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7f9a60072ac0 0x7f9a60074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a80c7a640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7f9a60072ac0 0x7f9a60074f40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.259+0000 7f9a8147b640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9a6006a7a0 0x7f9a6006cc60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a78033000 tx=0x7f9a78009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.260+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== osd.3 v2:172.21.15.49:6808/37909 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9a60075610 con 0x7f9a60072ac0 2024-03-14T22:05:30.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.272+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 --> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a7c06ab60 con 0x7f9a60072ac0 2024-03-14T22:05:30.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a71ffb640 1 -- 172.21.15.49:0/2187750447 <== osd.3 v2:172.21.15.49:6808/37909 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9a6c0099f0 con 0x7f9a60072ac0 2024-03-14T22:05:30.272 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:30.272 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:30.272 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7f9a60072ac0 msgr2=0x7f9a60074f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:6808/37909,v1:172.21.15.49:6809/37909] conn(0x7f9a60072ac0 0x7f9a60074f40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9a6006a7a0 msgr2=0x7f9a6006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f9a6006a7a0 0x7f9a6006cc60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a78033000 tx=0x7f9a78009030 comp rx=0 tx=0).stop 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1dfc90 msgr2=0x7f9a7c146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 --2- 172.21.15.49:0/2187750447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1dfc90 0x7f9a7c146eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c14f350 tx=0x7f9a74007b00 comp rx=0 tx=0).stop 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a81c7c640 1 -- 172.21.15.49:0/2187750447 reap_dead start 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 shutdown_connections 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 >> 172.21.15.49:0/2187750447 conn(0x7f9a7c0bcda0 msgr2=0x7f9a7c0bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 shutdown_connections 2024-03-14T22:05:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.273+0000 7f9a836df640 1 -- 172.21.15.49:0/2187750447 wait complete. 2024-03-14T22:05:30.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-14T22:05:30.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.390+0000 7f1e9ff9f640 1 Processor -- start 2024-03-14T22:05:30.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.390+0000 7f1e9ff9f640 1 -- start start 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9ff9f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 0x7f1e980bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9ff9f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e980b3a20 con 0x7f1e9814b820 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9ff9f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9806adf0 con 0x7f1e98153590 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9ff9f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980ba550 con 0x7f1e980bad60 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9dd3b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1e9814b820 0x7f1e9806a6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43644/0 (socket says 172.21.15.49:43644) 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9dd3b640 1 -- 172.21.15.49:0/170815614 learned_addr learned my addr 172.21.15.49:0/170815614 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900583720 0 0) 0x7f1e980b3a20 con 0x7f1e9814b820 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9d53a640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 0x7f1e980bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e84003520 con 0x7f1e9814b820 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832691485 0 0) 0x7f1e84003520 con 0x7f1e9814b820 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 msgr2=0x7f1e980bb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 0x7f1e980bb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 >> v1:172.21.15.97:6789/0 conn(0x7f1e98153590 legacy=0x7f1e98155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980bfbe0 con 0x7f1e9814b820 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f1e940019f0 con 0x7f1e9814b820 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 >> v1:172.21.15.49:6789/0 conn(0x7f1e9814b820 legacy=0x7f1e9806a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.391+0000 7f1e9cd39640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 0x7f1e840039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9cd39640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9cd39640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e84004d00 0x7f1e84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980ba550 con 0x7f1e84003f20 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8400ba60 con 0x7f1e84004d00 2024-03-14T22:05:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8400bc20 con 0x7f1e980bad60 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9dd3b640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 0x7f1e840039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9d53a640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9e53c640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e84004d00 0x7f1e84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9d53a640 1 -- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 msgr2=0x7f1e840039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9d53a640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980bad60 0x7f1e840039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9d53a640 1 -- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e84004d00 msgr2=0x7f1e84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9d53a640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e84004d00 0x7f1e84005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9d53a640 1 -- 172.21.15.49:0/170815614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84003520 con 0x7f1e84003f20 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.392+0000 7f1e9e53c640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e84004d00 0x7f1e84005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:30.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9d53a640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e840047c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009a90 tx=0x7f1e840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a624415693ecde1d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8802c680 con 0x7f1e84003f20 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8802c7e0 con 0x7f1e84003f20 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9cd39640 1 -- 172.21.15.49:0/170815614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8802cb90 con 0x7f1e84003f20 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/170815614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 msgr2=0x7f1e840047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9ff9f640 1 --2- 172.21.15.49:0/170815614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e840047c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009a90 tx=0x7f1e840057f0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9e53c640 1 -- 172.21.15.49:0/170815614 reap_dead start 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/170815614 shutdown_connections 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.393+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/170815614 >> 172.21.15.49:0/170815614 conn(0x7f1e980bcd80 msgr2=0x7f1e980bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.394+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/170815614 shutdown_connections 2024-03-14T22:05:30.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.394+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/170815614 wait complete. 2024-03-14T22:05:30.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 Processor -- start 2024-03-14T22:05:30.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 -- start start 2024-03-14T22:05:30.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e980a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980b4700 0x7f1e98144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e98145420 0x7f1e981e3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980a2920 con 0x7f1e84003f20 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980b5990 con 0x7f1e98145420 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9ff9f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e981e0090 con 0x7f1e980b4700 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9dd3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e980a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9dd3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e980a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50278/0 (socket says 172.21.15.49:50278) 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.395+0000 7f1e9dd3b640 1 -- 172.21.15.49:0/3261556280 learned_addr learned my addr 172.21.15.49:0/3261556280 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9dd3b640 1 -- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980b4700 msgr2=0x7f1e98144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9dd3b640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980b4700 0x7f1e98144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9dd3b640 1 -- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e98145420 msgr2=0x7f1e981e3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9dd3b640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e98145420 0x7f1e981e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9dd3b640 1 -- 172.21.15.49:0/3261556280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980b6670 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9dd3b640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e980a23e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007a10 tx=0x7f1e9400a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e8e7fc640 1 -- 172.21.15.49:0/3261556280 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e940133f0 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e8e7fc640 1 -- 172.21.15.49:0/3261556280 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e94013570 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e8e7fc640 1 -- 172.21.15.49:0/3261556280 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94013920 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e981458a0 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.396+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e980b68e0 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.397+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e98066bb0 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.397+0000 7f1e8e7fc640 1 -- 172.21.15.49:0/3261556280 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1e94013a80 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.398+0000 7f1e8e7fc640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f1e7c06a7a0 0x7f1e7c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.398+0000 7f1e8e7fc640 1 -- 172.21.15.49:0/3261556280 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1e94089b80 con 0x7f1e84003f20 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.398+0000 7f1e9d53a640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f1e7c06a7a0 0x7f1e7c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.399+0000 7f1e9d53a640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f1e7c06a7a0 0x7f1e7c06cc60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1e88036000 tx=0x7f1e8802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.402+0000 7f1e8e7fc640 1 -- 172.21.15.49:0/3261556280 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1e94054f20 con 0x7f1e84003f20 2024-03-14T22:05:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.581+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e9814bc00 con 0x7f1e7c06a7a0 2024-03-14T22:05:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.581+0000 7f1e8e7fc640 1 -- 172.21.15.49:0/3261556280 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1e8802c950 con 0x7f1e7c06a7a0 2024-03-14T22:05:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 3 to deep-scrub 2024-03-14T22:05:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f1e7c06a7a0 msgr2=0x7f1e7c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f1e7c06a7a0 0x7f1e7c06cc60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1e88036000 tx=0x7f1e8802e000 comp rx=0 tx=0).stop 2024-03-14T22:05:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 msgr2=0x7f1e980a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e980a23e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007a10 tx=0x7f1e9400a6e0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 shutdown_connections 2024-03-14T22:05:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f1e7c06a7a0 0x7f1e7c06cc60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e98145420 0x7f1e981e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e980b4700 0x7f1e98144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 --2- 172.21.15.49:0/3261556280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e84003f20 0x7f1e980a23e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 >> 172.21.15.49:0/3261556280 conn(0x7f1e980bcd80 msgr2=0x7f1e9809fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 shutdown_connections 2024-03-14T22:05:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.584+0000 7f1e9ff9f640 1 -- 172.21.15.49:0/3261556280 wait complete. 2024-03-14T22:05:30.594 INFO:tasks.ceph:Scrubbing osd.4 2024-03-14T22:05:30.594 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.706+0000 7f4bddeca640 1 Processor -- start 2024-03-14T22:05:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.706+0000 7f4bddeca640 1 -- start start 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.706+0000 7f4bddeca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bd80bfc80 0x7f4bd814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.706+0000 7f4bddeca640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bc940 0x7f4bd80bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.706+0000 7f4bddeca640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd806ae40 con 0x7f4bd80b9c00 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.706+0000 7f4bddeca640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd80ba750 con 0x7f4bd80bfc80 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.706+0000 7f4bddeca640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80bb550 con 0x7f4bd80bc940 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd6ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bc940 0x7f4bd80bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bd80bfc80 0x7f4bd814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4bd80b9c00 0x7f4bd814e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:52868/0 (socket says 172.21.15.49:52868) 2024-03-14T22:05:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd7fff640 1 -- 172.21.15.49:0/938477961 learned_addr learned my addr 172.21.15.49:0/938477961 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd67fc640 1 -- 172.21.15.49:0/938477961 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996173836 0 0) 0x7f4bd806ae40 con 0x7f4bd80b9c00 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd67fc640 1 -- 172.21.15.49:0/938477961 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc0003520 con 0x7f4bd80b9c00 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd6ffd640 1 -- 172.21.15.49:0/938477961 >> v2:172.21.15.97:3300/0 conn(0x7f4bd80bfc80 msgr2=0x7f4bd814ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd6ffd640 1 --2- 172.21.15.49:0/938477961 >> v2:172.21.15.97:3300/0 conn(0x7f4bd80bfc80 0x7f4bd814ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd6ffd640 1 -- 172.21.15.49:0/938477961 >> v1:172.21.15.97:6789/0 conn(0x7f4bd80b9c00 legacy=0x7f4bd814e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd6ffd640 1 -- 172.21.15.49:0/938477961 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80bc590 con 0x7f4bd80bc940 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.707+0000 7f4bd77fe640 1 --2- 172.21.15.49:0/938477961 >> v2:172.21.15.97:3300/0 conn(0x7f4bd80bfc80 0x7f4bd814ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.708+0000 7f4bd6ffd640 1 --2- 172.21.15.49:0/938477961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bc940 0x7f4bd80bb010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0097a0 tx=0x7f4bcc02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c512959ef9f2f799 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.708+0000 7f4bd67fc640 1 -- 172.21.15.49:0/938477961 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc02f050 con 0x7f4bd80bc940 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.708+0000 7f4bd67fc640 1 -- 172.21.15.49:0/938477961 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc031700 con 0x7f4bd80bc940 2024-03-14T22:05:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.708+0000 7f4bd67fc640 1 -- 172.21.15.49:0/938477961 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc031a70 con 0x7f4bd80bc940 2024-03-14T22:05:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 -- 172.21.15.49:0/938477961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bc940 msgr2=0x7f4bd80bb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 --2- 172.21.15.49:0/938477961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bc940 0x7f4bd80bb010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0097a0 tx=0x7f4bcc02b7a0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 -- 172.21.15.49:0/938477961 shutdown_connections 2024-03-14T22:05:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 --2- 172.21.15.49:0/938477961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bc940 0x7f4bd80bb010 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 --2- 172.21.15.49:0/938477961 >> v2:172.21.15.97:3300/0 conn(0x7f4bd80bfc80 0x7f4bd814ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 -- 172.21.15.49:0/938477961 >> 172.21.15.49:0/938477961 conn(0x7f4bd80bcda0 msgr2=0x7f4bd80bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 -- 172.21.15.49:0/938477961 shutdown_connections 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.709+0000 7f4bddeca640 1 -- 172.21.15.49:0/938477961 wait complete. 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.710+0000 7f4bddeca640 1 Processor -- start 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.710+0000 7f4bddeca640 1 -- start start 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.710+0000 7f4bddeca640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd80b9c00 0x7f4bd814e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.710+0000 7f4bddeca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd80bc940 0x7f4bd8099cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bddeca640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bfc80 0x7f4bd809a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bddeca640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd814ef20 con 0x7f4bd80b9c00 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bddeca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80b6c90 con 0x7f4bd80bc940 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bddeca640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd809b810 con 0x7f4bd80bfc80 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd80b9c00 0x7f4bd814e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd80b9c00 0x7f4bd814e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53400/0 (socket says 172.21.15.49:53400) 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 -- 172.21.15.49:0/1204117486 learned_addr learned my addr 172.21.15.49:0/1204117486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd7fff640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bfc80 0x7f4bd809a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd6ffd640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd80bc940 0x7f4bd8099cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 -- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bfc80 msgr2=0x7f4bd809a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bd80bfc80 0x7f4bd809a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 -- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd80bc940 msgr2=0x7f4bd8099cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd80bc940 0x7f4bd8099cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 -- 172.21.15.49:0/1204117486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009400 con 0x7f4bd80b9c00 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bd77fe640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd80b9c00 0x7f4bd814e9e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800e960 tx=0x7f4bc800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800c9a0 con 0x7f4bd80b9c00 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc009400 con 0x7f4bd80b9c00 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc800cb00 con 0x7f4bd80b9c00 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800ce90 con 0x7f4bd80b9c00 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.711+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd80a5430 con 0x7f4bd80b9c00 2024-03-14T22:05:30.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.712+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bd80c0060 con 0x7f4bd80b9c00 2024-03-14T22:05:30.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.713+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4bc8013470 con 0x7f4bd80b9c00 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.713+0000 7f4bbffff640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4bc406a7a0 0x7f4bc406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.713+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4bc8089080 con 0x7f4bd80b9c00 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.713+0000 7f4bd6ffd640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4bc406a7a0 0x7f4bc406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.713+0000 7f4bbffff640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7f4bc4072ac0 0x7f4bc4074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.713+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 --> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bc4075610 con 0x7f4bc4072ac0 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.713+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bc808b740 con 0x7f4bd80b9c00 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.714+0000 7f4bd7fff640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7f4bc4072ac0 0x7f4bc4074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.714+0000 7f4bd6ffd640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4bc406a7a0 0x7f4bc406cc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc035000 tx=0x7f4bcc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.714+0000 7f4bd7fff640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7f4bc4072ac0 0x7f4bc4074f40 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.715+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== osd.4 v2:172.21.15.97:6817/37904 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4bc4075610 con 0x7f4bc4072ac0 2024-03-14T22:05:30.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.728+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 --> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4bd806ab60 con 0x7f4bc4072ac0 2024-03-14T22:05:30.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.728+0000 7f4bbffff640 1 -- 172.21.15.49:0/1204117486 <== osd.4 v2:172.21.15.97:6817/37904 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4bd0002010 con 0x7f4bc4072ac0 2024-03-14T22:05:30.727 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:30.727 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:30.727 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:30.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.728+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7f4bc4072ac0 msgr2=0x7f4bc4074f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.728+0000 7f4bddeca640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6817/37904,v1:172.21.15.97:6819/37904] conn(0x7f4bc4072ac0 0x7f4bc4074f40 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4bc406a7a0 msgr2=0x7f4bc406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4bc406a7a0 0x7f4bc406cc60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc035000 tx=0x7f4bcc008030 comp rx=0 tx=0).stop 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd80b9c00 msgr2=0x7f4bd814e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 --2- 172.21.15.49:0/1204117486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd80b9c00 0x7f4bd814e9e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800e960 tx=0x7f4bc800ee20 comp rx=0 tx=0).stop 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bd7fff640 1 -- 172.21.15.49:0/1204117486 reap_dead start 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 shutdown_connections 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 >> 172.21.15.49:0/1204117486 conn(0x7f4bd80bcda0 msgr2=0x7f4bd80a0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 shutdown_connections 2024-03-14T22:05:30.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.729+0000 7f4bddeca640 1 -- 172.21.15.49:0/1204117486 wait complete. 2024-03-14T22:05:30.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-14T22:05:30.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.847+0000 7f2cf0b73640 1 Processor -- start 2024-03-14T22:05:30.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.847+0000 7f2cf0b73640 1 -- start start 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.847+0000 7f2cf0b73640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cec14b820 0x7f2cec06a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cf0b73640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec0bb1c0 0x7f2cec14e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cf0b73640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0b3a20 con 0x7f2cec0bad60 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cf0b73640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec0ba550 con 0x7f2cec14b820 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cf0b73640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0bfbe0 con 0x7f2cec0bb1c0 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cea59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cec14b820 0x7f2cec06a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cea59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cec14b820 0x7f2cec06a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53406/0 (socket says 172.21.15.49:53406) 2024-03-14T22:05:30.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2ce3fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2cec0bad60 0x7f2cec14e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44418/0 (socket says 172.21.15.49:44418) 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cea59c640 1 -- 172.21.15.49:0/1318573855 learned_addr learned my addr 172.21.15.49:0/1318573855 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cead9d640 1 --2- 172.21.15.49:0/1318573855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec0bb1c0 0x7f2cec14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cea59c640 1 -- 172.21.15.49:0/1318573855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec0bb1c0 msgr2=0x7f2cec14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cea59c640 1 --2- 172.21.15.49:0/1318573855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec0bb1c0 0x7f2cec14e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cea59c640 1 -- 172.21.15.49:0/1318573855 >> v1:172.21.15.49:6789/0 conn(0x7f2cec0bad60 legacy=0x7f2cec14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2cea59c640 1 -- 172.21.15.49:0/1318573855 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0bfec0 con 0x7f2cec14b820 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.848+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/1318573855 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739103172 0 0) 0x7f2cec0b3a20 con 0x7f2cec0bad60 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.849+0000 7f2cea59c640 1 --2- 172.21.15.49:0/1318573855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec14b820 0x7f2cec06a6e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009870 tx=0x7f2cdc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc85e9eaa56bff90 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.849+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/1318573855 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc030840 con 0x7f2cec14b820 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.849+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/1318573855 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc0309a0 con 0x7f2cec14b820 2024-03-14T22:05:30.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.849+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/1318573855 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc030d30 con 0x7f2cec14b820 2024-03-14T22:05:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.849+0000 7f2cf0b73640 1 -- 172.21.15.49:0/1318573855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec14b820 msgr2=0x7f2cec06a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.849+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/1318573855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec14b820 0x7f2cec06a6e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009870 tx=0x7f2cdc02b980 comp rx=0 tx=0).stop 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.849+0000 7f2cf0b73640 1 -- 172.21.15.49:0/1318573855 shutdown_connections 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.850+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/1318573855 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec0bb1c0 0x7f2cec14e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.850+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/1318573855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec14b820 0x7f2cec06a6e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.850+0000 7f2cf0b73640 1 -- 172.21.15.49:0/1318573855 >> 172.21.15.49:0/1318573855 conn(0x7f2cec0bcd80 msgr2=0x7f2cec0bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.850+0000 7f2cf0b73640 1 -- 172.21.15.49:0/1318573855 shutdown_connections 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.850+0000 7f2cf0b73640 1 -- 172.21.15.49:0/1318573855 wait complete. 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 Processor -- start 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 -- start start 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0bad60 0x7f2cec0b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec0bb1c0 0x7f2cec099d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec14b820 0x7f2cec09e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0b6d10 con 0x7f2cec0bad60 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec1e3cc0 con 0x7f2cec0bb1c0 2024-03-14T22:05:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cf0b73640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec148d00 con 0x7f2cec14b820 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0bad60 0x7f2cec0b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0bad60 0x7f2cec0b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53412/0 (socket says 172.21.15.49:53412) 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.851+0000 7f2cea59c640 1 -- 172.21.15.49:0/2076074479 learned_addr learned my addr 172.21.15.49:0/2076074479 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cead9d640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec14b820 0x7f2cec09e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2ce3fff640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec0bb1c0 0x7f2cec099d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cea59c640 1 -- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec14b820 msgr2=0x7f2cec09e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cea59c640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec14b820 0x7f2cec09e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cea59c640 1 -- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec0bb1c0 msgr2=0x7f2cec099d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cea59c640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec0bb1c0 0x7f2cec099d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cea59c640 1 -- 172.21.15.49:0/2076074479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0094d0 con 0x7f2cec0bad60 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cea59c640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0bad60 0x7f2cec0b67d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000c00 tx=0x7f2cdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2ce2ffd640 1 -- 172.21.15.49:0/2076074479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc03e040 con 0x7f2cec0bad60 2024-03-14T22:05:30.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2ce2ffd640 1 -- 172.21.15.49:0/2076074479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc039c90 con 0x7f2cec0bad60 2024-03-14T22:05:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec1e2f00 con 0x7f2cec0bad60 2024-03-14T22:05:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2ce2ffd640 1 -- 172.21.15.49:0/2076074479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc004030 con 0x7f2cec0bad60 2024-03-14T22:05:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.852+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec1e34c0 con 0x7f2cec0bad60 2024-03-14T22:05:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.853+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec066bb0 con 0x7f2cec0bad60 2024-03-14T22:05:30.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.853+0000 7f2ce2ffd640 1 -- 172.21.15.49:0/2076074479 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2cdc004190 con 0x7f2cec0bad60 2024-03-14T22:05:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.854+0000 7f2ce2ffd640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2ccc06a7d0 0x7f2ccc06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.854+0000 7f2ce2ffd640 1 -- 172.21.15.49:0/2076074479 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2cdc0ab450 con 0x7f2cec0bad60 2024-03-14T22:05:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.854+0000 7f2ce3fff640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2ccc06a7d0 0x7f2ccc06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:30.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.855+0000 7f2ce3fff640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2ccc06a7d0 0x7f2ccc06cc90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2cec06b8b0 tx=0x7f2cd4007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:30.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:30.858+0000 7f2ce2ffd640 1 -- 172.21.15.49:0/2076074479 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2cdc076880 con 0x7f2cec0bad60 2024-03-14T22:05:31.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.037+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cec14ad70 con 0x7f2ccc06a7d0 2024-03-14T22:05:31.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.038+0000 7f2ce2ffd640 1 -- 172.21.15.49:0/2076074479 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2cd4007270 con 0x7f2ccc06a7d0 2024-03-14T22:05:31.037 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 4 to deep-scrub 2024-03-14T22:05:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2ccc06a7d0 msgr2=0x7f2ccc06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2ccc06a7d0 0x7f2ccc06cc90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2cec06b8b0 tx=0x7f2cd4007c00 comp rx=0 tx=0).stop 2024-03-14T22:05:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0bad60 msgr2=0x7f2cec0b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0bad60 0x7f2cec0b67d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000c00 tx=0x7f2cdc004730 comp rx=0 tx=0).stop 2024-03-14T22:05:31.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 shutdown_connections 2024-03-14T22:05:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2ccc06a7d0 0x7f2ccc06cc90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cec14b820 0x7f2cec09e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cec0bb1c0 0x7f2cec099d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 --2- 172.21.15.49:0/2076074479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0bad60 0x7f2cec0b67d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 >> 172.21.15.49:0/2076074479 conn(0x7f2cec0bcd80 msgr2=0x7f2cec0a1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 shutdown_connections 2024-03-14T22:05:31.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.040+0000 7f2cf0b73640 1 -- 172.21.15.49:0/2076074479 wait complete. 2024-03-14T22:05:31.050 INFO:tasks.ceph:Scrubbing osd.5 2024-03-14T22:05:31.050 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:31.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.167+0000 7f48bcee9640 1 Processor -- start 2024-03-14T22:05:31.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.167+0000 7f48bcee9640 1 -- start start 2024-03-14T22:05:31.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.167+0000 7f48bcee9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 0x7f48b8147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48bcee9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b8147a40 con 0x7f48b80b8eb0 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48bcee9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b8142010 con 0x7f48b8148410 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48bcee9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b806e690 con 0x7f48b806c870 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b659c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f48b80b8eb0 0x7f48b8141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44430/0 (socket says 172.21.15.49:44430) 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b659c640 1 -- 172.21.15.49:0/3499935467 learned_addr learned my addr 172.21.15.49:0/3499935467 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b5d9b640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 0x7f48b8147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904179962 0 0) 0x7f48b8147a40 con 0x7f48b80b8eb0 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a0003520 con 0x7f48b80b8eb0 2024-03-14T22:05:31.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3239766862 0 0) 0x7f48a0003520 con 0x7f48b80b8eb0 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 msgr2=0x7f48b8147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 0x7f48b8147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 >> v1:172.21.15.97:6789/0 conn(0x7f48b8148410 legacy=0x7f48b81520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b806ccb0 con 0x7f48b80b8eb0 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f48a8001670 con 0x7f48b80b8eb0 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 >> v1:172.21.15.49:6789/0 conn(0x7f48b80b8eb0 legacy=0x7f48b8141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a0003a80 0x7f48a0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a00047c0 0x7f48a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 0x7f48a0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b806e690 con 0x7f48a0003a80 2024-03-14T22:05:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a000bb10 con 0x7f48a00047c0 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a000bca0 con 0x7f48b806c870 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b659c640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a0003a80 0x7f48a0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b5d9b640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a00047c0 0x7f48a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.168+0000 7f48b6d9d640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 0x7f48a0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.169+0000 7f48b659c640 1 -- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 msgr2=0x7f48a0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.169+0000 7f48b659c640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b806c870 0x7f48a0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.169+0000 7f48b659c640 1 -- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a00047c0 msgr2=0x7f48a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.169+0000 7f48b659c640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a00047c0 0x7f48a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.169+0000 7f48b659c640 1 -- 172.21.15.49:0/3499935467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0003520 con 0x7f48a0003a80 2024-03-14T22:05:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.169+0000 7f48b659c640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a0003a80 0x7f48a0003ea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48a800c240 tx=0x7f48a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d063d0d150cf46c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8005ad0 con 0x7f48a0003a80 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a8005c30 con 0x7f48a0003a80 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48b559a640 1 -- 172.21.15.49:0/3499935467 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a802c8b0 con 0x7f48a0003a80 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48bcee9640 1 -- 172.21.15.49:0/3499935467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a0003a80 msgr2=0x7f48a0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48bcee9640 1 --2- 172.21.15.49:0/3499935467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48a0003a80 0x7f48a0003ea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f48a800c240 tx=0x7f48a00057f0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48b6d9d640 1 -- 172.21.15.49:0/3499935467 reap_dead start 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48bcee9640 1 -- 172.21.15.49:0/3499935467 shutdown_connections 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48bcee9640 1 -- 172.21.15.49:0/3499935467 >> 172.21.15.49:0/3499935467 conn(0x7f48b80b9d30 msgr2=0x7f48b80ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48bcee9640 1 -- 172.21.15.49:0/3499935467 shutdown_connections 2024-03-14T22:05:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.170+0000 7f48bcee9640 1 -- 172.21.15.49:0/3499935467 wait complete. 2024-03-14T22:05:31.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 Processor -- start 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 -- start start 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a0003a80 0x7f48b80b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b8145d50 0x7f48b80b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80a1e50 0x7f48b81e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80bcfe0 con 0x7f48b80a1e50 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80b2da0 con 0x7f48a0003a80 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48bcee9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80b4050 con 0x7f48b8145d50 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48b6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80a1e50 0x7f48b81e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48b6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80a1e50 0x7f48b81e0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53422/0 (socket says 172.21.15.49:53422) 2024-03-14T22:05:31.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48b6d9d640 1 -- 172.21.15.49:0/2538192100 learned_addr learned my addr 172.21.15.49:0/2538192100 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:31.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48b5d9b640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b8145d50 0x7f48b80b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.172+0000 7f48b659c640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a0003a80 0x7f48b80b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48b6d9d640 1 -- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b8145d50 msgr2=0x7f48b80b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48b6d9d640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48b8145d50 0x7f48b80b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48b6d9d640 1 -- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a0003a80 msgr2=0x7f48b80b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48b6d9d640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a0003a80 0x7f48b80b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48b6d9d640 1 -- 172.21.15.49:0/2538192100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80bcd00 con 0x7f48b80a1e50 2024-03-14T22:05:31.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48b6d9d640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80a1e50 0x7f48b81e0fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48b806c700 tx=0x7f48b000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0006d60 con 0x7f48b80a1e50 2024-03-14T22:05:31.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0003970 con 0x7f48b80a1e50 2024-03-14T22:05:31.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0003d20 con 0x7f48b80a1e50 2024-03-14T22:05:31.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b80a1370 con 0x7f48b80a1e50 2024-03-14T22:05:31.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.173+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b80a1930 con 0x7f48b80a1e50 2024-03-14T22:05:31.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.174+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48b806d500 con 0x7f48b80a1e50 2024-03-14T22:05:31.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.174+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f48b0005070 con 0x7f48b80a1e50 2024-03-14T22:05:31.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48a6ffd640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f489806a7d0 0x7f489806cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f48b0088230 con 0x7f48b80a1e50 2024-03-14T22:05:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48a6ffd640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7f4898072af0 0x7f4898074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48b659c640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f489806a7d0 0x7f489806cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 --> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4898075640 con 0x7f4898072af0 2024-03-14T22:05:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48b008a8f0 con 0x7f48b80a1e50 2024-03-14T22:05:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48b5d9b640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7f4898072af0 0x7f4898074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.175+0000 7f48b659c640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f489806a7d0 0x7f489806cc90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48a8034000 tx=0x7f48a8009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.176+0000 7f48b5d9b640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7f4898072af0 0x7f4898074f70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.176+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== osd.5 v2:172.21.15.97:6816/37921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4898075640 con 0x7f4898072af0 2024-03-14T22:05:31.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.189+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 --> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f48b806d220 con 0x7f4898072af0 2024-03-14T22:05:31.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48a6ffd640 1 -- 172.21.15.49:0/2538192100 <== osd.5 v2:172.21.15.97:6816/37921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f48ac009b50 con 0x7f4898072af0 2024-03-14T22:05:31.189 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:31.190 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:31.190 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:31.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7f4898072af0 msgr2=0x7f4898074f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6816/37921,v1:172.21.15.97:6818/37921] conn(0x7f4898072af0 0x7f4898074f70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f489806a7d0 msgr2=0x7f489806cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f489806a7d0 0x7f489806cc90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48a8034000 tx=0x7f48a8009030 comp rx=0 tx=0).stop 2024-03-14T22:05:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80a1e50 msgr2=0x7f48b81e0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 --2- 172.21.15.49:0/2538192100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48b80a1e50 0x7f48b81e0fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48b806c700 tx=0x7f48b000fd40 comp rx=0 tx=0).stop 2024-03-14T22:05:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48b6d9d640 1 -- 172.21.15.49:0/2538192100 reap_dead start 2024-03-14T22:05:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 shutdown_connections 2024-03-14T22:05:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 >> 172.21.15.49:0/2538192100 conn(0x7f48b80b9d30 msgr2=0x7f48b813e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 shutdown_connections 2024-03-14T22:05:31.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.190+0000 7f48bcee9640 1 -- 172.21.15.49:0/2538192100 wait complete. 2024-03-14T22:05:31.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-14T22:05:31.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.307+0000 7f8f7f1a9640 1 Processor -- start 2024-03-14T22:05:31.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.307+0000 7f8f7f1a9640 1 -- start start 2024-03-14T22:05:31.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7f1a9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f780bb040 0x7f8f7814b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7f1a9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f7806da70 0x7f8f780b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7f1a9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f7806e160 0x7f8f7806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7f1a9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f780b98e0 con 0x7f8f7806e160 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7f1a9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7814aa20 con 0x7f8f7806da70 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7f1a9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780bf300 con 0x7f8f780bb040 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7d746640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f7806e160 0x7f8f7806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7d746640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f7806e160 0x7f8f7806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53432/0 (socket says 172.21.15.49:53432) 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7d746640 1 -- 172.21.15.49:0/2177309760 learned_addr learned my addr 172.21.15.49:0/2177309760 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f7cf45640 1 --2- 172.21.15.49:0/2177309760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f780bb040 0x7f8f7814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.308+0000 7f8f77fff640 1 --2- 172.21.15.49:0/2177309760 >> v2:172.21.15.97:3300/0 conn(0x7f8f7806da70 0x7f8f780b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f7d746640 1 -- 172.21.15.49:0/2177309760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f780bb040 msgr2=0x7f8f7814b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f7d746640 1 --2- 172.21.15.49:0/2177309760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f780bb040 0x7f8f7814b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f7d746640 1 -- 172.21.15.49:0/2177309760 >> v2:172.21.15.97:3300/0 conn(0x7f8f7806da70 msgr2=0x7f8f780b8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f7d746640 1 --2- 172.21.15.49:0/2177309760 >> v2:172.21.15.97:3300/0 conn(0x7f8f7806da70 0x7f8f780b8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f7d746640 1 -- 172.21.15.49:0/2177309760 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7814b970 con 0x7f8f7806e160 2024-03-14T22:05:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f77fff640 1 --2- 172.21.15.49:0/2177309760 >> v2:172.21.15.97:3300/0 conn(0x7f8f7806da70 0x7f8f780b8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:31.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f7cf45640 1 --2- 172.21.15.49:0/2177309760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f780bb040 0x7f8f7814b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:31.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f7d746640 1 --2- 172.21.15.49:0/2177309760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f7806e160 0x7f8f7806e5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806ac20 tx=0x7f8f6802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1059085348ff750f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f777fe640 1 -- 172.21.15.49:0/2177309760 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68030860 con 0x7f8f7806e160 2024-03-14T22:05:31.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f777fe640 1 -- 172.21.15.49:0/2177309760 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f680309c0 con 0x7f8f7806e160 2024-03-14T22:05:31.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.309+0000 7f8f777fe640 1 -- 172.21.15.49:0/2177309760 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68030d50 con 0x7f8f7806e160 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/2177309760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f7806e160 msgr2=0x7f8f7806e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/2177309760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f7806e160 0x7f8f7806e5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806ac20 tx=0x7f8f6802b9a0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/2177309760 shutdown_connections 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/2177309760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f7806e160 0x7f8f7806e5f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/2177309760 >> v2:172.21.15.97:3300/0 conn(0x7f8f7806da70 0x7f8f780b8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/2177309760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f780bb040 0x7f8f7814b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/2177309760 >> 172.21.15.49:0/2177309760 conn(0x7f8f780be400 msgr2=0x7f8f780be810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/2177309760 shutdown_connections 2024-03-14T22:05:31.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.310+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/2177309760 wait complete. 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.311+0000 7f8f7f1a9640 1 Processor -- start 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.311+0000 7f8f7f1a9640 1 -- start start 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7f1a9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f7806da70 0x7f8f781e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7f1a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7806e160 0x7f8f7813f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7f1a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f780bb040 0x7f8f7809f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7f1a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7813e490 con 0x7f8f780bb040 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7f1a9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7809fbc0 con 0x7f8f7806e160 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7f1a9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f781dff40 con 0x7f8f7806da70 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f780bb040 0x7f8f7809f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f780bb040 0x7f8f7809f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53438/0 (socket says 172.21.15.49:53438) 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 -- 172.21.15.49:0/931907436 learned_addr learned my addr 172.21.15.49:0/931907436 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7cf45640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f7806da70 0x7f8f781e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f77fff640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7806e160 0x7f8f7813f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 -- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f7806da70 msgr2=0x7f8f781e09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f7806da70 0x7f8f781e09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 -- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7806e160 msgr2=0x7f8f7813f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7806e160 0x7f8f7813f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.312+0000 7f8f7d746640 1 -- 172.21.15.49:0/931907436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f680094d0 con 0x7f8f780bb040 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.313+0000 7f8f7d746640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f780bb040 0x7f8f7809f680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f680090b0 tx=0x7f8f68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.313+0000 7f8f757fa640 1 -- 172.21.15.49:0/931907436 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68039d50 con 0x7f8f780bb040 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.313+0000 7f8f757fa640 1 -- 172.21.15.49:0/931907436 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f68043400 con 0x7f8f780bb040 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.313+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f781e01e0 con 0x7f8f780bb040 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.313+0000 7f8f757fa640 1 -- 172.21.15.49:0/931907436 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f680437b0 con 0x7f8f780bb040 2024-03-14T22:05:31.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.313+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7809fe40 con 0x7f8f780bb040 2024-03-14T22:05:31.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.314+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f781e0700 con 0x7f8f780bb040 2024-03-14T22:05:31.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.314+0000 7f8f757fa640 1 -- 172.21.15.49:0/931907436 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8f68043910 con 0x7f8f780bb040 2024-03-14T22:05:31.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.315+0000 7f8f757fa640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f6406a7d0 0x7f8f6406cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.315+0000 7f8f757fa640 1 -- 172.21.15.49:0/931907436 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8f6803e070 con 0x7f8f780bb040 2024-03-14T22:05:31.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.315+0000 7f8f7cf45640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f6406a7d0 0x7f8f6406cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.315+0000 7f8f7cf45640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f6406a7d0 0x7f8f6406cc90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009fd0 tx=0x7f8f70009870 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.319+0000 7f8f757fa640 1 -- 172.21.15.49:0/931907436 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8f781e0700 con 0x7f8f780bb040 2024-03-14T22:05:31.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.498+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f7814ac30 con 0x7f8f6406a7d0 2024-03-14T22:05:31.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.499+0000 7f8f757fa640 1 -- 172.21.15.49:0/931907436 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8f7000f040 con 0x7f8f6406a7d0 2024-03-14T22:05:31.498 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 5 to deep-scrub 2024-03-14T22:05:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f6406a7d0 msgr2=0x7f8f6406cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f6406a7d0 0x7f8f6406cc90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009fd0 tx=0x7f8f70009870 comp rx=0 tx=0).stop 2024-03-14T22:05:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f780bb040 msgr2=0x7f8f7809f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f780bb040 0x7f8f7809f680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f680090b0 tx=0x7f8f68004730 comp rx=0 tx=0).stop 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 shutdown_connections 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f8f6406a7d0 0x7f8f6406cc90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f780bb040 0x7f8f7809f680 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f7806e160 0x7f8f7813f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 --2- 172.21.15.49:0/931907436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f7806da70 0x7f8f781e09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 >> 172.21.15.49:0/931907436 conn(0x7f8f780be400 msgr2=0x7f8f780be7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 shutdown_connections 2024-03-14T22:05:31.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.501+0000 7f8f7f1a9640 1 -- 172.21.15.49:0/931907436 wait complete. 2024-03-14T22:05:31.511 INFO:tasks.ceph:Scrubbing osd.6 2024-03-14T22:05:31.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:31.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.622+0000 7f15f33b9640 1 Processor -- start 2024-03-14T22:05:31.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.622+0000 7f15f33b9640 1 -- start start 2024-03-14T22:05:31.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15f33b9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec144f90 0x7f15ec14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15f33b9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ec14af80 con 0x7f15ec0b9e60 2024-03-14T22:05:31.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15f33b9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ec0b8e00 con 0x7f15ec071270 2024-03-14T22:05:31.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15f33b9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0b9300 con 0x7f15ec144f90 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15f1956640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f15ec0b9e60 0x7f15ec1552a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44436/0 (socket says 172.21.15.49:44436) 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15f1956640 1 -- 172.21.15.49:0/4172730939 learned_addr learned my addr 172.21.15.49:0/4172730939 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15f1155640 1 --2- 172.21.15.49:0/4172730939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec144f90 0x7f15ec14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.623+0000 7f15e3fff640 1 -- 172.21.15.49:0/4172730939 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514823352 0 0) 0x7f15ec14af80 con 0x7f15ec0b9e60 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.624+0000 7f15e3fff640 1 -- 172.21.15.49:0/4172730939 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15d8003520 con 0x7f15ec0b9e60 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.624+0000 7f15e3fff640 1 -- 172.21.15.49:0/4172730939 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999284438 0 0) 0x7f15ec0b8e00 con 0x7f15ec071270 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.624+0000 7f15e3fff640 1 -- 172.21.15.49:0/4172730939 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15ec14af80 con 0x7f15ec071270 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.624+0000 7f15f1155640 1 -- 172.21.15.49:0/4172730939 >> v1:172.21.15.97:6789/0 conn(0x7f15ec071270 legacy=0x7f15ec0b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.624+0000 7f15f1155640 1 -- 172.21.15.49:0/4172730939 >> v1:172.21.15.49:6789/0 conn(0x7f15ec0b9e60 legacy=0x7f15ec1552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.624+0000 7f15f1155640 1 -- 172.21.15.49:0/4172730939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec14b1f0 con 0x7f15ec144f90 2024-03-14T22:05:31.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.624+0000 7f15f1155640 1 --2- 172.21.15.49:0/4172730939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec144f90 0x7f15ec14aa40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009dc0 tx=0x7f15e802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb206f6db082f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15e3fff640 1 -- 172.21.15.49:0/4172730939 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e802ccd0 con 0x7f15ec144f90 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15e3fff640 1 -- 172.21.15.49:0/4172730939 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e802ce30 con 0x7f15ec144f90 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15e3fff640 1 -- 172.21.15.49:0/4172730939 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8034660 con 0x7f15ec144f90 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15f33b9640 1 -- 172.21.15.49:0/4172730939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec144f90 msgr2=0x7f15ec14aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15f33b9640 1 --2- 172.21.15.49:0/4172730939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec144f90 0x7f15ec14aa40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009dc0 tx=0x7f15e802bf20 comp rx=0 tx=0).stop 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15f33b9640 1 -- 172.21.15.49:0/4172730939 shutdown_connections 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15f33b9640 1 --2- 172.21.15.49:0/4172730939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec144f90 0x7f15ec14aa40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15f33b9640 1 -- 172.21.15.49:0/4172730939 >> 172.21.15.49:0/4172730939 conn(0x7f15ec0a01b0 msgr2=0x7f15ec0a0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15f33b9640 1 -- 172.21.15.49:0/4172730939 shutdown_connections 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.625+0000 7f15f33b9640 1 -- 172.21.15.49:0/4172730939 wait complete. 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.626+0000 7f15f33b9640 1 Processor -- start 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.626+0000 7f15f33b9640 1 -- start start 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f33b9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec071270 0x7f15ec13df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f33b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ec0b9e60 0x7f15ec0bd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f33b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec144f90 0x7f15ec13b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f33b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0bde40 con 0x7f15ec144f90 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f33b9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec09f460 con 0x7f15ec0b9e60 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f33b9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec09ebc0 con 0x7f15ec071270 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec144f90 0x7f15ec13b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec144f90 0x7f15ec13b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53440/0 (socket says 172.21.15.49:53440) 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 -- 172.21.15.49:0/2710551092 learned_addr learned my addr 172.21.15.49:0/2710551092 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1155640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec071270 0x7f15ec13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f0954640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ec0b9e60 0x7f15ec0bd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 -- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec071270 msgr2=0x7f15ec13df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec071270 0x7f15ec13df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 -- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ec0b9e60 msgr2=0x7f15ec0bd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ec0b9e60 0x7f15ec0bd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1956640 1 -- 172.21.15.49:0/2710551092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8009a70 con 0x7f15ec144f90 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f1155640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15ec071270 0x7f15ec13df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:31.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.627+0000 7f15f0954640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ec0b9e60 0x7f15ec0bd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:31.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.628+0000 7f15f1956640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec144f90 0x7f15ec13b770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c430 tx=0x7f15e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.628+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4005ce0 con 0x7f15ec144f90 2024-03-14T22:05:31.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.628+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4005e40 con 0x7f15ec144f90 2024-03-14T22:05:31.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.628+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec09ee10 con 0x7f15ec144f90 2024-03-14T22:05:31.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.628+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e40072f0 con 0x7f15ec144f90 2024-03-14T22:05:31.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.628+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec09dfa0 con 0x7f15ec144f90 2024-03-14T22:05:31.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.629+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15ec14b7b0 con 0x7f15ec144f90 2024-03-14T22:05:31.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.629+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f15e4007450 con 0x7f15ec144f90 2024-03-14T22:05:31.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15e1ffb640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f15d006a7d0 0x7f15d006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f15e4088370 con 0x7f15ec144f90 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15e1ffb640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f15d0072af0 0x7f15d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 --> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15d0075640 con 0x7f15d0072af0 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e408aa30 con 0x7f15ec144f90 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15f1155640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f15d006a7d0 0x7f15d006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15f0954640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f15d0072af0 0x7f15d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.630+0000 7f15f1155640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f15d006a7d0 0x7f15d006cc90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f15e8036000 tx=0x7f15e8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.631+0000 7f15f0954640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f15d0072af0 0x7f15d0074f70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.631+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== osd.6 v2:172.21.15.97:6800/37902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f15d0075640 con 0x7f15d0072af0 2024-03-14T22:05:31.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.644+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 --> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f15ec09f610 con 0x7f15d0072af0 2024-03-14T22:05:31.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.644+0000 7f15e1ffb640 1 -- 172.21.15.49:0/2710551092 <== osd.6 v2:172.21.15.97:6800/37902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f15ec09f610 con 0x7f15d0072af0 2024-03-14T22:05:31.643 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:31.643 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:31.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.644+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f15d0072af0 msgr2=0x7f15d0074f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.644+0000 7f15f33b9640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6800/37902,v1:172.21.15.97:6801/37902] conn(0x7f15d0072af0 0x7f15d0074f70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f15d006a7d0 msgr2=0x7f15d006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f15d006a7d0 0x7f15d006cc90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f15e8036000 tx=0x7f15e8030000 comp rx=0 tx=0).stop 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec144f90 msgr2=0x7f15ec13b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 --2- 172.21.15.49:0/2710551092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec144f90 0x7f15ec13b770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c430 tx=0x7f15e400c8f0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f1956640 1 -- 172.21.15.49:0/2710551092 reap_dead start 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 shutdown_connections 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 >> 172.21.15.49:0/2710551092 conn(0x7f15ec0a01b0 msgr2=0x7f15ec0b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 shutdown_connections 2024-03-14T22:05:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.645+0000 7f15f33b9640 1 -- 172.21.15.49:0/2710551092 wait complete. 2024-03-14T22:05:31.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-14T22:05:31.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.763+0000 7fd76ffea640 1 Processor -- start 2024-03-14T22:05:31.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.763+0000 7fd76ffea640 1 -- start start 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76ffea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd76806a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76ffea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7680bc780 0x7fd76814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76ffea640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76814b540 con 0x7fd76814b0b0 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76ffea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76806b960 con 0x7fd7680bc780 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76ffea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680bb020 con 0x7fd768144db0 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76e587640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd76814b0b0 0x7fd76814dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44450/0 (socket says 172.21.15.49:44450) 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76e587640 1 -- 172.21.15.49:0/4141742864 learned_addr learned my addr 172.21.15.49:0/4141742864 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76d585640 1 --2- 172.21.15.49:0/4141742864 >> v2:172.21.15.97:3300/0 conn(0x7fd7680bc780 0x7fd76814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76dd86640 1 --2- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd76806a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76cd84640 1 -- 172.21.15.49:0/4141742864 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637559152 0 0) 0x7fd76814b540 con 0x7fd76814b0b0 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76cd84640 1 -- 172.21.15.49:0/4141742864 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd754003520 con 0x7fd76814b0b0 2024-03-14T22:05:31.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76d585640 1 -- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 msgr2=0x7fd76806a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:31.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76d585640 1 --2- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd76806a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76d585640 1 -- 172.21.15.49:0/4141742864 >> v1:172.21.15.49:6789/0 conn(0x7fd76814b0b0 legacy=0x7fd76814dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76d585640 1 -- 172.21.15.49:0/4141742864 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76806b3a0 con 0x7fd7680bc780 2024-03-14T22:05:31.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.764+0000 7fd76dd86640 1 --2- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd76806a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T22:05:31.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.765+0000 7fd76d585640 1 --2- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7680bc780 0x7fd76814ab70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd7580097a0 tx=0x7fd75802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72252c1984518933 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.765+0000 7fd76cd84640 1 -- 172.21.15.49:0/4141742864 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75802e070 con 0x7fd7680bc780 2024-03-14T22:05:31.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.765+0000 7fd76cd84640 1 -- 172.21.15.49:0/4141742864 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7580315a0 con 0x7fd7680bc780 2024-03-14T22:05:31.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76cd84640 1 -- 172.21.15.49:0/4141742864 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758031910 con 0x7fd7680bc780 2024-03-14T22:05:31.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 -- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7680bc780 msgr2=0x7fd76814ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 --2- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7680bc780 0x7fd76814ab70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd7580097a0 tx=0x7fd75802b840 comp rx=0 tx=0).stop 2024-03-14T22:05:31.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 -- 172.21.15.49:0/4141742864 shutdown_connections 2024-03-14T22:05:31.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 --2- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7680bc780 0x7fd76814ab70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 --2- 172.21.15.49:0/4141742864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd76806a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 -- 172.21.15.49:0/4141742864 >> 172.21.15.49:0/4141742864 conn(0x7fd7680c0200 msgr2=0x7fd7680c0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 -- 172.21.15.49:0/4141742864 shutdown_connections 2024-03-14T22:05:31.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.766+0000 7fd76ffea640 1 -- 172.21.15.49:0/4141742864 wait complete. 2024-03-14T22:05:31.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.767+0000 7fd76ffea640 1 Processor -- start 2024-03-14T22:05:31.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.767+0000 7fd76ffea640 1 -- start start 2024-03-14T22:05:31.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76ffea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7680bc780 0x7fd7680a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76ffea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd7680a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76ffea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76814b0b0 0x7fd7680a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76ffea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680a5490 con 0x7fd7680bc780 2024-03-14T22:05:31.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76ffea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680a3230 con 0x7fd76814b0b0 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76ffea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd768145760 con 0x7fd768144db0 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7680bc780 0x7fd7680a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7680bc780 0x7fd7680a4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53450/0 (socket says 172.21.15.49:53450) 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 -- 172.21.15.49:0/3422609503 learned_addr learned my addr 172.21.15.49:0/3422609503 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76e587640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76814b0b0 0x7fd7680a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76d585640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd7680a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 -- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 msgr2=0x7fd7680a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd7680a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 -- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76814b0b0 msgr2=0x7fd7680a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76814b0b0 0x7fd7680a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 -- 172.21.15.49:0/3422609503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd764009a70 con 0x7fd7680bc780 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.768+0000 7fd76dd86640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7680bc780 0x7fd7680a4f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd76400efc0 tx=0x7fd76400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.769+0000 7fd75e7fc640 1 -- 172.21.15.49:0/3422609503 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764014680 con 0x7fd7680bc780 2024-03-14T22:05:31.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.769+0000 7fd75e7fc640 1 -- 172.21.15.49:0/3422609503 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76400f040 con 0x7fd7680bc780 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.769+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758009400 con 0x7fd7680bc780 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.769+0000 7fd75e7fc640 1 -- 172.21.15.49:0/3422609503 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764014a00 con 0x7fd7680bc780 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.769+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7681e3fa0 con 0x7fd7680bc780 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.770+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7680a2ef0 con 0x7fd7680bc780 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.770+0000 7fd75e7fc640 1 -- 172.21.15.49:0/3422609503 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd764014b60 con 0x7fd7680bc780 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.770+0000 7fd75e7fc640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd74c06a7a0 0x7fd74c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.770+0000 7fd75e7fc640 1 -- 172.21.15.49:0/3422609503 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd764089ca0 con 0x7fd7680bc780 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.771+0000 7fd76d585640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd74c06a7a0 0x7fd74c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:31.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.771+0000 7fd76d585640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd74c06a7a0 0x7fd74c06cc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd758036000 tx=0x7fd758008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:31.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.775+0000 7fd75e7fc640 1 -- 172.21.15.49:0/3422609503 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd764055180 con 0x7fd7680bc780 2024-03-14T22:05:31.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.954+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fd768145190 con 0x7fd74c06a7a0 2024-03-14T22:05:31.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.954+0000 7fd75e7fc640 1 -- 172.21.15.49:0/3422609503 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd758006e10 con 0x7fd74c06a7a0 2024-03-14T22:05:31.953 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 6 to deep-scrub 2024-03-14T22:05:31.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd74c06a7a0 msgr2=0x7fd74c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd74c06a7a0 0x7fd74c06cc60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd758036000 tx=0x7fd758008030 comp rx=0 tx=0).stop 2024-03-14T22:05:31.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7680bc780 msgr2=0x7fd7680a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7680bc780 0x7fd7680a4f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd76400efc0 tx=0x7fd76400c490 comp rx=0 tx=0).stop 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 shutdown_connections 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd74c06a7a0 0x7fd74c06cc60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76814b0b0 0x7fd7680a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd768144db0 0x7fd7680a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 --2- 172.21.15.49:0/3422609503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7680bc780 0x7fd7680a4f50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 >> 172.21.15.49:0/3422609503 conn(0x7fd7680c0200 msgr2=0x7fd768072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 shutdown_connections 2024-03-14T22:05:31.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:31.956+0000 7fd76ffea640 1 -- 172.21.15.49:0/3422609503 wait complete. 2024-03-14T22:05:31.966 INFO:tasks.ceph:Scrubbing osd.7 2024-03-14T22:05:31.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-14T22:05:32.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.077+0000 7f2cb41bf640 1 Processor -- start 2024-03-14T22:05:32.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.077+0000 7f2cb41bf640 1 -- start start 2024-03-14T22:05:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb41bf640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cac0bfc80 0x7f2cac14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb41bf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2cac0bc940 0x7f2cac0bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb41bf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0b9c00 0x7f2cac0ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb41bf640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0ba750 con 0x7f2cac0bfc80 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb41bf640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0bb550 con 0x7f2cac0bc940 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb41bf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac06ae40 con 0x7f2cac0b9c00 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cac0bfc80 0x7f2cac14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cac0bfc80 0x7f2cac14ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53452/0 (socket says 172.21.15.49:53452) 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 -- 172.21.15.49:0/2037799306 learned_addr learned my addr 172.21.15.49:0/2037799306 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb175a640 1 --2- 172.21.15.49:0/2037799306 >> v2:172.21.15.97:3300/0 conn(0x7f2cac0bc940 0x7f2cac0bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb275c640 1 --2- 172.21.15.49:0/2037799306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0b9c00 0x7f2cac0ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 -- 172.21.15.49:0/2037799306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0b9c00 msgr2=0x7f2cac0ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 --2- 172.21.15.49:0/2037799306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0b9c00 0x7f2cac0ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 -- 172.21.15.49:0/2037799306 >> v2:172.21.15.97:3300/0 conn(0x7f2cac0bc940 msgr2=0x7f2cac0bb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 --2- 172.21.15.49:0/2037799306 >> v2:172.21.15.97:3300/0 conn(0x7f2cac0bc940 0x7f2cac0bb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb1f5b640 1 -- 172.21.15.49:0/2037799306 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0bc590 con 0x7f2cac0bfc80 2024-03-14T22:05:32.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.078+0000 7f2cb175a640 1 --2- 172.21.15.49:0/2037799306 >> v2:172.21.15.97:3300/0 conn(0x7f2cac0bc940 0x7f2cac0bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.079+0000 7f2cb1f5b640 1 --2- 172.21.15.49:0/2037799306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14ad90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2cac06a3c0 tx=0x7f2ca802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849d1ccdbb29cf79 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.079+0000 7f2cb0f59640 1 -- 172.21.15.49:0/2037799306 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8031750 con 0x7f2cac0bfc80 2024-03-14T22:05:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.079+0000 7f2cb0f59640 1 -- 172.21.15.49:0/2037799306 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca80318b0 con 0x7f2cac0bfc80 2024-03-14T22:05:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.079+0000 7f2cb0f59640 1 -- 172.21.15.49:0/2037799306 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8031c40 con 0x7f2cac0bfc80 2024-03-14T22:05:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 -- 172.21.15.49:0/2037799306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 msgr2=0x7f2cac14ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 --2- 172.21.15.49:0/2037799306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14ad90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2cac06a3c0 tx=0x7f2ca802b7f0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 -- 172.21.15.49:0/2037799306 shutdown_connections 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 --2- 172.21.15.49:0/2037799306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0b9c00 0x7f2cac0ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 --2- 172.21.15.49:0/2037799306 >> v2:172.21.15.97:3300/0 conn(0x7f2cac0bc940 0x7f2cac0bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 --2- 172.21.15.49:0/2037799306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14ad90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 -- 172.21.15.49:0/2037799306 >> 172.21.15.49:0/2037799306 conn(0x7f2cac0bcda0 msgr2=0x7f2cac0bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 -- 172.21.15.49:0/2037799306 shutdown_connections 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.080+0000 7f2cb41bf640 1 -- 172.21.15.49:0/2037799306 wait complete. 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 Processor -- start 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 -- start start 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cac0b9c00 0x7f2cac0b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0bc940 0x7f2cac14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0b6b50 con 0x7f2cac0bfc80 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac14c160 con 0x7f2cac0b9c00 2024-03-14T22:05:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb41bf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac14f1b0 con 0x7f2cac0bc940 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb275c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb275c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53458/0 (socket says 172.21.15.49:53458) 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.081+0000 7f2cb275c640 1 -- 172.21.15.49:0/1272429591 learned_addr learned my addr 172.21.15.49:0/1272429591 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb1f5b640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cac0b9c00 0x7f2cac0b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb175a640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0bc940 0x7f2cac14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb275c640 1 -- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0bc940 msgr2=0x7f2cac14e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb275c640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2cac0bc940 0x7f2cac14e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb275c640 1 -- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cac0b9c00 msgr2=0x7f2cac0b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb275c640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cac0b9c00 0x7f2cac0b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb275c640 1 -- 172.21.15.49:0/1272429591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80094d0 con 0x7f2cac0bfc80 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb275c640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4007e60 tx=0x7f2ca400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4014620 con 0x7f2cac0bfc80 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca40147a0 con 0x7f2cac0bfc80 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac14e590 con 0x7f2cac0bfc80 2024-03-14T22:05:32.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4014b50 con 0x7f2cac0bfc80 2024-03-14T22:05:32.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.082+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac14ec10 con 0x7f2cac0bfc80 2024-03-14T22:05:32.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.083+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cac0c0060 con 0x7f2cac0bfc80 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2ca4014cb0 con 0x7f2cac0bfc80 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2ca27fc640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2c9006a7a0 0x7f2c9006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2ca4089750 con 0x7f2cac0bfc80 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2ca27fc640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f2c90072ac0 0x7f2c90074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 --> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c90075610 con 0x7f2c90072ac0 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2cb1f5b640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2c9006a7a0 0x7f2c9006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca408be10 con 0x7f2cac0bfc80 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.084+0000 7f2cb175a640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f2c90072ac0 0x7f2c90074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.085+0000 7f2cb175a640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f2c90072ac0 0x7f2c90074f40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.085+0000 7f2cb1f5b640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2c9006a7a0 0x7f2c9006cc60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8035000 tx=0x7f2ca8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.085+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== osd.7 v2:172.21.15.97:6808/37903 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2c90075610 con 0x7f2c90072ac0 2024-03-14T22:05:32.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.098+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 --> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2cac06ab60 con 0x7f2c90072ac0 2024-03-14T22:05:32.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.098+0000 7f2ca27fc640 1 -- 172.21.15.49:0/1272429591 <== osd.7 v2:172.21.15.97:6808/37903 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2c9c00ca30 con 0x7f2c90072ac0 2024-03-14T22:05:32.098 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-14T22:05:32.098 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T22:05:32.098 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-14T22:05:32.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f2c90072ac0 msgr2=0x7f2c90074f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6808/37903,v1:172.21.15.97:6809/37903] conn(0x7f2c90072ac0 0x7f2c90074f40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2c9006a7a0 msgr2=0x7f2c9006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f2c9006a7a0 0x7f2c9006cc60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8035000 tx=0x7f2ca8008030 comp rx=0 tx=0).stop 2024-03-14T22:05:32.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 msgr2=0x7f2cac14d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 --2- 172.21.15.49:0/1272429591 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cac0bfc80 0x7f2cac14d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4007e60 tx=0x7f2ca400e4b0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb275c640 1 -- 172.21.15.49:0/1272429591 reap_dead start 2024-03-14T22:05:32.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 shutdown_connections 2024-03-14T22:05:32.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 >> 172.21.15.49:0/1272429591 conn(0x7f2cac0bcda0 msgr2=0x7f2cac0a05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:32.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 shutdown_connections 2024-03-14T22:05:32.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.099+0000 7f2cb41bf640 1 -- 172.21.15.49:0/1272429591 wait complete. 2024-03-14T22:05:32.109 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-14T22:05:32.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.218+0000 7f74f944b640 1 Processor -- start 2024-03-14T22:05:32.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.218+0000 7f74f944b640 1 -- start start 2024-03-14T22:05:32.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.218+0000 7f74f944b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f74f414b820 0x7f74f406a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.218+0000 7f74f944b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f40bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.218+0000 7f74f944b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f40b3a20 con 0x7f74f414e270 2024-03-14T22:05:32.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.218+0000 7f74f944b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f40ba550 con 0x7f74f414b820 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.218+0000 7f74f944b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40bfbe0 con 0x7f74f40bad60 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f27fc640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f40bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f74f414b820 0x7f74f406a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f74f414e270 0x7f74f414e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:52898/0 (socket says 172.21.15.49:52898) 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f37fe640 1 -- 172.21.15.49:0/1012183508 learned_addr learned my addr 172.21.15.49:0/1012183508 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f27fc640 1 -- 172.21.15.49:0/1012183508 >> v2:172.21.15.97:3300/0 conn(0x7f74f414b820 msgr2=0x7f74f406a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f27fc640 1 --2- 172.21.15.49:0/1012183508 >> v2:172.21.15.97:3300/0 conn(0x7f74f414b820 0x7f74f406a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f27fc640 1 -- 172.21.15.49:0/1012183508 >> v1:172.21.15.97:6789/0 conn(0x7f74f414e270 legacy=0x7f74f414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.219+0000 7f74f27fc640 1 -- 172.21.15.49:0/1012183508 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f40bfec0 con 0x7f74f40bad60 2024-03-14T22:05:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.220+0000 7f74f27fc640 1 --2- 172.21.15.49:0/1012183508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f40bb1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74e80097a0 tx=0x7f74e802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527491bdfc47b619 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.220+0000 7f74f1ffb640 1 -- 172.21.15.49:0/1012183508 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8031830 con 0x7f74f40bad60 2024-03-14T22:05:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.220+0000 7f74f1ffb640 1 -- 172.21.15.49:0/1012183508 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e8031990 con 0x7f74f40bad60 2024-03-14T22:05:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.220+0000 7f74f1ffb640 1 -- 172.21.15.49:0/1012183508 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8031d20 con 0x7f74f40bad60 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 -- 172.21.15.49:0/1012183508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 msgr2=0x7f74f40bb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 --2- 172.21.15.49:0/1012183508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f40bb1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74e80097a0 tx=0x7f74e802b8d0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 -- 172.21.15.49:0/1012183508 shutdown_connections 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 --2- 172.21.15.49:0/1012183508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f40bb1c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 --2- 172.21.15.49:0/1012183508 >> v2:172.21.15.97:3300/0 conn(0x7f74f414b820 0x7f74f406a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 -- 172.21.15.49:0/1012183508 >> 172.21.15.49:0/1012183508 conn(0x7f74f40bcd80 msgr2=0x7f74f40bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 -- 172.21.15.49:0/1012183508 shutdown_connections 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.221+0000 7f74f944b640 1 -- 172.21.15.49:0/1012183508 wait complete. 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.222+0000 7f74f944b640 1 Processor -- start 2024-03-14T22:05:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.222+0000 7f74f944b640 1 -- start start 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.222+0000 7f74f944b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f41d3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.222+0000 7f74f944b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f414b820 0x7f74f41d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.222+0000 7f74f944b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f414e270 0x7f74f41d5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f944b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f41d5750 con 0x7f74f414b820 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f944b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f41e0530 con 0x7f74f414e270 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f944b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f41dfc40 con 0x7f74f40bad60 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f414b820 0x7f74f41d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f414b820 0x7f74f41d4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53474/0 (socket says 172.21.15.49:53474) 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 -- 172.21.15.49:0/433890671 learned_addr learned my addr 172.21.15.49:0/433890671 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f37fe640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f414e270 0x7f74f41d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f2ffd640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f41d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 -- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 msgr2=0x7f74f41d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f41d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 -- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f414e270 msgr2=0x7f74f41d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f414e270 0x7f74f41d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 -- 172.21.15.49:0/433890671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8009400 con 0x7f74f414b820 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f2ffd640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f41d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f27fc640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f414b820 0x7f74f41d4cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74e80098c0 tx=0x7f74e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74db7fe640 1 -- 172.21.15.49:0/433890671 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e803a6f0 con 0x7f74f414b820 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74db7fe640 1 -- 172.21.15.49:0/433890671 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e803a870 con 0x7f74f414b820 2024-03-14T22:05:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f40b3c70 con 0x7f74f414b820 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74db7fe640 1 -- 172.21.15.49:0/433890671 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e803ac20 con 0x7f74f414b820 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.223+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f41e01c0 con 0x7f74f414b820 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.224+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f41e06a0 con 0x7f74f414b820 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.225+0000 7f74db7fe640 1 -- 172.21.15.49:0/433890671 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f74e804c3b0 con 0x7f74f414b820 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.225+0000 7f74db7fe640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f74e006a7a0 0x7f74e006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.225+0000 7f74db7fe640 1 -- 172.21.15.49:0/433890671 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f74e80aab10 con 0x7f74f414b820 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.225+0000 7f74f2ffd640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f74e006a7a0 0x7f74e006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.226+0000 7f74f2ffd640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f74e006a7a0 0x7f74e006cc60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f74e40074c0 tx=0x7f74e400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.229+0000 7f74db7fe640 1 -- 172.21.15.49:0/433890671 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f74e8076060 con 0x7f74f414b820 2024-03-14T22:05:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.409+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f74f414bc00 con 0x7f74e006a7a0 2024-03-14T22:05:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.409+0000 7f74db7fe640 1 -- 172.21.15.49:0/433890671 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f74e400f050 con 0x7f74e006a7a0 2024-03-14T22:05:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 7 to deep-scrub 2024-03-14T22:05:32.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f74e006a7a0 msgr2=0x7f74e006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f74e006a7a0 0x7f74e006cc60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f74e40074c0 tx=0x7f74e400b040 comp rx=0 tx=0).stop 2024-03-14T22:05:32.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f414b820 msgr2=0x7f74f41d4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f414b820 0x7f74f41d4cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f74e80098c0 tx=0x7f74e80046c0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 shutdown_connections 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f74e006a7a0 0x7f74e006cc60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74f414e270 0x7f74f41d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f414b820 0x7f74f41d4cd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 --2- 172.21.15.49:0/433890671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74f40bad60 0x7f74f41d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 >> 172.21.15.49:0/433890671 conn(0x7f74f40bcd80 msgr2=0x7f74f40b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 shutdown_connections 2024-03-14T22:05:32.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.411+0000 7f74f944b640 1 -- 172.21.15.49:0/433890671 wait complete. 2024-03-14T22:05:32.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T22:05:32.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 Processor -- start 2024-03-14T22:05:32.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 -- start start 2024-03-14T22:05:32.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840baef0 0x7f10841480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f108406b620 0x7f10840bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f10840bc940 0x7f10840bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108406ad10 con 0x7f10840bc940 2024-03-14T22:05:32.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108406a6e0 con 0x7f108406b620 2024-03-14T22:05:32.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.529+0000 7f108c147640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10840bc2b0 con 0x7f10840baef0 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f10840bc940 0x7f10840bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f10840bc940 0x7f10840bfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53478/0 (socket says 172.21.15.49:53478) 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 -- 172.21.15.49:0/86456129 learned_addr learned my addr 172.21.15.49:0/86456129 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f1089ee3640 1 --2- 172.21.15.49:0/86456129 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840baef0 0x7f10841480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 -- 172.21.15.49:0/86456129 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840baef0 msgr2=0x7f10841480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f10896e2640 1 --2- 172.21.15.49:0/86456129 >> v2:172.21.15.97:3300/0 conn(0x7f108406b620 0x7f10840bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 --2- 172.21.15.49:0/86456129 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840baef0 0x7f10841480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 -- 172.21.15.49:0/86456129 >> v2:172.21.15.97:3300/0 conn(0x7f108406b620 msgr2=0x7f10840bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 --2- 172.21.15.49:0/86456129 >> v2:172.21.15.97:3300/0 conn(0x7f108406b620 0x7f10840bf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.530+0000 7f108a6e4640 1 -- 172.21.15.49:0/86456129 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108406bab0 con 0x7f10840bc940 2024-03-14T22:05:32.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108a6e4640 1 --2- 172.21.15.49:0/86456129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840bc940 0x7f10840bfd30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1084069f10 tx=0x7f107c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9981975fafee4ac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f1088ee1640 1 -- 172.21.15.49:0/86456129 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c02c9e0 con 0x7f10840bc940 2024-03-14T22:05:32.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f1088ee1640 1 -- 172.21.15.49:0/86456129 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c02cb40 con 0x7f10840bc940 2024-03-14T22:05:32.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f1088ee1640 1 -- 172.21.15.49:0/86456129 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0343f0 con 0x7f10840bc940 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 -- 172.21.15.49:0/86456129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840bc940 msgr2=0x7f10840bfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 --2- 172.21.15.49:0/86456129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840bc940 0x7f10840bfd30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1084069f10 tx=0x7f107c02be40 comp rx=0 tx=0).stop 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 -- 172.21.15.49:0/86456129 shutdown_connections 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 --2- 172.21.15.49:0/86456129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840bc940 0x7f10840bfd30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 --2- 172.21.15.49:0/86456129 >> v2:172.21.15.97:3300/0 conn(0x7f108406b620 0x7f10840bf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 --2- 172.21.15.49:0/86456129 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840baef0 0x7f10841480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 -- 172.21.15.49:0/86456129 >> 172.21.15.49:0/86456129 conn(0x7f10840bcd80 msgr2=0x7f10840bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 -- 172.21.15.49:0/86456129 shutdown_connections 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.531+0000 7f108c147640 1 -- 172.21.15.49:0/86456129 wait complete. 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.532+0000 7f108c147640 1 Processor -- start 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.532+0000 7f108c147640 1 -- start start 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f108c147640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840baef0 0x7f108414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f108c147640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840bc940 0x7f108414ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f108c147640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108414cf50 0x7f10840b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f108c147640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108414eae0 con 0x7f10840baef0 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f108c147640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108414d400 con 0x7f108414cf50 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f108c147640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10841e00f0 con 0x7f10840bc940 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840baef0 0x7f108414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840baef0 0x7f108414e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53484/0 (socket says 172.21.15.49:53484) 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 -- 172.21.15.49:0/1498019563 learned_addr learned my addr 172.21.15.49:0/1498019563 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f10896e2640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840bc940 0x7f108414ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f108a6e4640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108414cf50 0x7f10840b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 -- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840bc940 msgr2=0x7f108414ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840bc940 0x7f108414ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 -- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108414cf50 msgr2=0x7f10840b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108414cf50 0x7f10840b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f1089ee3640 1 -- 172.21.15.49:0/1498019563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c009b30 con 0x7f10840baef0 2024-03-14T22:05:32.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.533+0000 7f10896e2640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840bc940 0x7f108414ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:05:32.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.534+0000 7f1089ee3640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840baef0 0x7f108414e5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f108000c820 tx=0x7f108000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.534+0000 7f107a7fc640 1 -- 172.21.15.49:0/1498019563 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080004220 con 0x7f10840baef0 2024-03-14T22:05:32.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.534+0000 7f107a7fc640 1 -- 172.21.15.49:0/1498019563 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10800043a0 con 0x7f10840baef0 2024-03-14T22:05:32.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.534+0000 7f107a7fc640 1 -- 172.21.15.49:0/1498019563 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080007a70 con 0x7f10840baef0 2024-03-14T22:05:32.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.534+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10841e0370 con 0x7f10840baef0 2024-03-14T22:05:32.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.534+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10841e0930 con 0x7f10840baef0 2024-03-14T22:05:32.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.535+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1084066bb0 con 0x7f10840baef0 2024-03-14T22:05:32.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.535+0000 7f107a7fc640 1 -- 172.21.15.49:0/1498019563 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1080007bd0 con 0x7f10840baef0 2024-03-14T22:05:32.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.536+0000 7f107a7fc640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f106806a7a0 0x7f106806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:32.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.536+0000 7f107a7fc640 1 -- 172.21.15.49:0/1498019563 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1080088c90 con 0x7f10840baef0 2024-03-14T22:05:32.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.536+0000 7f10896e2640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f106806a7a0 0x7f106806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:32.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.536+0000 7f10896e2640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f106806a7a0 0x7f106806cc60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f108414ed40 tx=0x7f1074008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:32.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.540+0000 7f107a7fc640 1 -- 172.21.15.49:0/1498019563 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1080053fa0 con 0x7f10840baef0 2024-03-14T22:05:32.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.715+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10840bb2d0 con 0x7f106806a7a0 2024-03-14T22:05:32.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.723+0000 7f107a7fc640 1 -- 172.21.15.49:0/1498019563 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291065 (secure 0 0 0) 0x7f107400c050 con 0x7f106806a7a0 2024-03-14T22:05:32.725 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:05:32.727 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T22:05:32.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f106806a7a0 msgr2=0x7f106806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f106806a7a0 0x7f106806cc60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f108414ed40 tx=0x7f1074008040 comp rx=0 tx=0).stop 2024-03-14T22:05:32.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840baef0 msgr2=0x7f108414e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840baef0 0x7f108414e5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f108000c820 tx=0x7f108000cce0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 shutdown_connections 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f106806a7a0 0x7f106806cc60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108414cf50 0x7f10840b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10840bc940 0x7f108414ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 --2- 172.21.15.49:0/1498019563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10840baef0 0x7f108414e5a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 >> 172.21.15.49:0/1498019563 conn(0x7f10840bcd80 msgr2=0x7f10840b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 shutdown_connections 2024-03-14T22:05:32.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:32.726+0000 7f108c147640 1 -- 172.21.15.49:0/1498019563 wait complete. 2024-03-14T22:05:32.736 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":1567,"stamp":"2024-03-14T22:05:31.590129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257497981,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":4402,"num_read_kb":112073,"num_write":843876,"num_write_kb":9407945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896104,"ondisk_log_size":896104,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8155636,"kb_used_data":504884,"kb_used_omap":13,"kb_used_meta":7650674,"kb_avail":746819084,"statfs":{"total":773094113280,"available":764742742016,"internally_reserved":0,"allocated":517001216,"data_stored":515697415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":7834290264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15738128,"num_objects":-1200,"num_object_clones":0,"num_object_copies":-2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1200,"num_whiteouts":0,"num_read":12,"num_read_kb":140,"num_write":1406,"num_write_kb":3377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003065"},"pg_stats":[{"pgid":"3.3f","version":"23'10331","reported_seq":10373,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.730720+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T22:05:25.730720+0000","last_peered":"2024-03-14T22:05:25.730720+0000","last_clean":"2024-03-14T22:05:25.730720+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T22:05:25.730720+0000","last_undegraded":"2024-03-14T22:05:25.730720+0000","last_fullsized":"2024-03-14T22:05:25.730720+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:57:09.325211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8785,"num_write_kb":106962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'9098","reported_seq":9148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.747335+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T22:05:25.747335+0000","last_peered":"2024-03-14T22:05:25.747335+0000","last_clean":"2024-03-14T22:05:25.747335+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T22:05:25.747335+0000","last_undegraded":"2024-03-14T22:05:25.747335+0000","last_fullsized":"2024-03-14T22:05:25.747335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":7584,"num_write_kb":103945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'11564","reported_seq":11606,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.712807+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T22:05:25.712807+0000","last_peered":"2024-03-14T22:05:25.712807+0000","last_clean":"2024-03-14T22:05:25.712807+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T22:05:25.712807+0000","last_undegraded":"2024-03-14T22:05:25.712807+0000","last_fullsized":"2024-03-14T22:05:25.712807+0000","mapping_epoch":19,"log_start":"23'1500","ondisk_log_start":"23'1500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1274,"num_write":9914,"num_write_kb":127294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'10102","reported_seq":10148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.796834+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T22:05:25.796834+0000","last_peered":"2024-03-14T22:05:25.796834+0000","last_clean":"2024-03-14T22:05:25.796834+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T22:05:25.796834+0000","last_undegraded":"2024-03-14T22:05:25.796834+0000","last_fullsized":"2024-03-14T22:05:25.796834+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8407,"num_write_kb":110879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'9608","reported_seq":9642,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.708406+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T22:05:25.708406+0000","last_peered":"2024-03-14T22:05:25.708406+0000","last_clean":"2024-03-14T22:05:25.708406+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T22:05:25.708406+0000","last_undegraded":"2024-03-14T22:05:25.708406+0000","last_fullsized":"2024-03-14T22:05:25.708406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9608,"log_dups_size":0,"ondisk_log_size":9608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8008,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'9645","reported_seq":9685,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.884998+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T22:05:25.884998+0000","last_peered":"2024-03-14T22:05:25.884998+0000","last_clean":"2024-03-14T22:05:25.884998+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T22:05:25.884998+0000","last_undegraded":"2024-03-14T22:05:25.884998+0000","last_fullsized":"2024-03-14T22:05:25.884998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9645,"log_dups_size":0,"ondisk_log_size":9645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":8054,"num_write_kb":114907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10037","reported_seq":10073,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.757333+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T22:05:25.757333+0000","last_peered":"2024-03-14T22:05:25.757333+0000","last_clean":"2024-03-14T22:05:25.757333+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T22:05:25.757333+0000","last_undegraded":"2024-03-14T22:05:25.757333+0000","last_fullsized":"2024-03-14T22:05:25.757333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":8423,"num_write_kb":104789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'11196","reported_seq":11236,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.858919+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T22:05:25.858919+0000","last_peered":"2024-03-14T22:05:25.858919+0000","last_clean":"2024-03-14T22:05:25.858919+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T22:05:25.858919+0000","last_undegraded":"2024-03-14T22:05:25.858919+0000","last_fullsized":"2024-03-14T22:05:25.858919+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":9605,"num_write_kb":125652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'9017","reported_seq":9054,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.829992+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T22:05:25.829992+0000","last_peered":"2024-03-14T22:05:25.829992+0000","last_clean":"2024-03-14T22:05:25.829992+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T22:05:25.829992+0000","last_undegraded":"2024-03-14T22:05:25.829992+0000","last_fullsized":"2024-03-14T22:05:25.829992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9017,"log_dups_size":0,"ondisk_log_size":9017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:27:43.623783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7381,"num_write_kb":104568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'9539","reported_seq":9575,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.836191+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T22:05:25.836191+0000","last_peered":"2024-03-14T22:05:25.836191+0000","last_clean":"2024-03-14T22:05:25.836191+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T22:05:25.836191+0000","last_undegraded":"2024-03-14T22:05:25.836191+0000","last_fullsized":"2024-03-14T22:05:25.836191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7843,"num_write_kb":113016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'9229","reported_seq":9270,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.805062+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T22:05:25.805062+0000","last_peered":"2024-03-14T22:05:25.805062+0000","last_clean":"2024-03-14T22:05:25.805062+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T22:05:25.805062+0000","last_undegraded":"2024-03-14T22:05:25.805062+0000","last_fullsized":"2024-03-14T22:05:25.805062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9229,"log_dups_size":0,"ondisk_log_size":9229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7677,"num_write_kb":104837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'10311","reported_seq":10346,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.921059+0000","last_change":"2024-03-14T21:14:03.034043+0000","last_active":"2024-03-14T22:05:25.921059+0000","last_peered":"2024-03-14T22:05:25.921059+0000","last_clean":"2024-03-14T22:05:25.921059+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T22:05:25.921059+0000","last_undegraded":"2024-03-14T22:05:25.921059+0000","last_fullsized":"2024-03-14T22:05:25.921059+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:33.787920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8730,"num_write_kb":105734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'8864","reported_seq":8902,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.791971+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T22:05:25.791971+0000","last_peered":"2024-03-14T22:05:25.791971+0000","last_clean":"2024-03-14T22:05:25.791971+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T22:05:25.791971+0000","last_undegraded":"2024-03-14T22:05:25.791971+0000","last_fullsized":"2024-03-14T22:05:25.791971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8864,"log_dups_size":0,"ondisk_log_size":8864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":7264,"num_write_kb":95419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'10416","reported_seq":10456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.821241+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T22:05:25.821241+0000","last_peered":"2024-03-14T22:05:25.821241+0000","last_clean":"2024-03-14T22:05:25.821241+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T22:05:25.821241+0000","last_undegraded":"2024-03-14T22:05:25.821241+0000","last_fullsized":"2024-03-14T22:05:25.821241+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8793,"num_write_kb":111405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'9704","reported_seq":9743,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.901656+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T22:05:25.901656+0000","last_peered":"2024-03-14T22:05:25.901656+0000","last_clean":"2024-03-14T22:05:25.901656+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T22:05:25.901656+0000","last_undegraded":"2024-03-14T22:05:25.901656+0000","last_fullsized":"2024-03-14T22:05:25.901656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9704,"log_dups_size":0,"ondisk_log_size":9704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8012,"num_write_kb":95820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'9661","reported_seq":9709,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.804689+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T22:05:25.804689+0000","last_peered":"2024-03-14T22:05:25.804689+0000","last_clean":"2024-03-14T22:05:25.804689+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T22:05:25.804689+0000","last_undegraded":"2024-03-14T22:05:25.804689+0000","last_fullsized":"2024-03-14T22:05:25.804689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9661,"log_dups_size":0,"ondisk_log_size":9661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3572,"num_write":8076,"num_write_kb":107857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8662","reported_seq":8699,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.697094+0000","last_change":"2024-03-14T21:14:44.859268+0000","last_active":"2024-03-14T22:05:25.697094+0000","last_peered":"2024-03-14T22:05:25.697094+0000","last_clean":"2024-03-14T22:05:25.697094+0000","last_became_active":"2024-03-14T21:14:44.858906+0000","last_became_peered":"2024-03-14T21:14:44.858906+0000","last_unstale":"2024-03-14T22:05:25.697094+0000","last_undegraded":"2024-03-14T22:05:25.697094+0000","last_fullsized":"2024-03-14T22:05:25.697094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8662,"log_dups_size":0,"ondisk_log_size":8662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:56:07.747779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1528,"num_write":7020,"num_write_kb":102847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10199","reported_seq":10235,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.754730+0000","last_change":"2024-03-14T21:14:03.029823+0000","last_active":"2024-03-14T22:05:25.754730+0000","last_peered":"2024-03-14T22:05:25.754730+0000","last_clean":"2024-03-14T22:05:25.754730+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T22:05:25.754730+0000","last_undegraded":"2024-03-14T22:05:25.754730+0000","last_fullsized":"2024-03-14T22:05:25.754730+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:47.131679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8552,"num_write_kb":105487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'10521","reported_seq":10555,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.822188+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T22:05:25.822188+0000","last_peered":"2024-03-14T22:05:25.822188+0000","last_clean":"2024-03-14T22:05:25.822188+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T22:05:25.822188+0000","last_undegraded":"2024-03-14T22:05:25.822188+0000","last_fullsized":"2024-03-14T22:05:25.822188+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":118219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'10900","reported_seq":10938,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.858536+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T22:05:25.858536+0000","last_peered":"2024-03-14T22:05:25.858536+0000","last_clean":"2024-03-14T22:05:25.858536+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T22:05:25.858536+0000","last_undegraded":"2024-03-14T22:05:25.858536+0000","last_fullsized":"2024-03-14T22:05:25.858536+0000","mapping_epoch":19,"log_start":"23'800","ondisk_log_start":"23'800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:19:52.643816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":9315,"num_write_kb":117297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'10962","reported_seq":11009,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.883758+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T22:05:25.883758+0000","last_peered":"2024-03-14T22:05:25.883758+0000","last_clean":"2024-03-14T22:05:25.883758+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T22:05:25.883758+0000","last_undegraded":"2024-03-14T22:05:25.883758+0000","last_fullsized":"2024-03-14T22:05:25.883758+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3050,"num_write":9338,"num_write_kb":109552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10723","reported_seq":10759,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.871652+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T22:05:25.871652+0000","last_peered":"2024-03-14T22:05:25.871652+0000","last_clean":"2024-03-14T22:05:25.871652+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T22:05:25.871652+0000","last_undegraded":"2024-03-14T22:05:25.871652+0000","last_fullsized":"2024-03-14T22:05:25.871652+0000","mapping_epoch":19,"log_start":"23'700","ondisk_log_start":"23'700","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":9066,"num_write_kb":118743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'10565","reported_seq":10601,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.887877+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T22:05:25.887877+0000","last_peered":"2024-03-14T22:05:25.887877+0000","last_clean":"2024-03-14T22:05:25.887877+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T22:05:25.887877+0000","last_undegraded":"2024-03-14T22:05:25.887877+0000","last_fullsized":"2024-03-14T22:05:25.887877+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8950,"num_write_kb":109690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'11180","reported_seq":11221,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.799965+0000","last_change":"2024-03-14T21:14:03.038694+0000","last_active":"2024-03-14T22:05:25.799965+0000","last_peered":"2024-03-14T22:05:25.799965+0000","last_clean":"2024-03-14T22:05:25.799965+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T22:05:25.799965+0000","last_undegraded":"2024-03-14T22:05:25.799965+0000","last_fullsized":"2024-03-14T22:05:25.799965+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:03:10.586081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1530,"num_write":9571,"num_write_kb":127549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'10195","reported_seq":10232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.859354+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T22:05:25.859354+0000","last_peered":"2024-03-14T22:05:25.859354+0000","last_clean":"2024-03-14T22:05:25.859354+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T22:05:25.859354+0000","last_undegraded":"2024-03-14T22:05:25.859354+0000","last_fullsized":"2024-03-14T22:05:25.859354+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":8582,"num_write_kb":110246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'12334","reported_seq":12375,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.835792+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T22:05:25.835792+0000","last_peered":"2024-03-14T22:05:25.835792+0000","last_clean":"2024-03-14T22:05:25.835792+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T22:05:25.835792+0000","last_undegraded":"2024-03-14T22:05:25.835792+0000","last_fullsized":"2024-03-14T22:05:25.835792+0000","mapping_epoch":19,"log_start":"23'2300","ondisk_log_start":"23'2300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:39:04.947869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10697,"num_write_kb":121303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'9243","reported_seq":9287,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.671635+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T22:05:25.671635+0000","last_peered":"2024-03-14T22:05:25.671635+0000","last_clean":"2024-03-14T22:05:25.671635+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T22:05:25.671635+0000","last_undegraded":"2024-03-14T22:05:25.671635+0000","last_fullsized":"2024-03-14T22:05:25.671635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7601,"num_write_kb":100407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'10312","reported_seq":10347,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.888629+0000","last_change":"2024-03-14T21:14:03.040690+0000","last_active":"2024-03-14T22:05:25.888629+0000","last_peered":"2024-03-14T22:05:25.888629+0000","last_clean":"2024-03-14T22:05:25.888629+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T22:05:25.888629+0000","last_undegraded":"2024-03-14T22:05:25.888629+0000","last_fullsized":"2024-03-14T22:05:25.888629+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":108137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'11957","reported_seq":12006,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.874501+0000","last_change":"2024-03-14T21:14:44.746590+0000","last_active":"2024-03-14T22:05:25.874501+0000","last_peered":"2024-03-14T22:05:25.874501+0000","last_clean":"2024-03-14T22:05:25.874501+0000","last_became_active":"2024-03-14T21:14:44.746271+0000","last_became_peered":"2024-03-14T21:14:44.746271+0000","last_unstale":"2024-03-14T22:05:25.874501+0000","last_undegraded":"2024-03-14T22:05:25.874501+0000","last_fullsized":"2024-03-14T22:05:25.874501+0000","mapping_epoch":22,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":10306,"num_write_kb":120667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'11954","reported_seq":11996,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.885100+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T22:05:25.885100+0000","last_peered":"2024-03-14T22:05:25.885100+0000","last_clean":"2024-03-14T22:05:25.885100+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T22:05:25.885100+0000","last_undegraded":"2024-03-14T22:05:25.885100+0000","last_fullsized":"2024-03-14T22:05:25.885100+0000","mapping_epoch":19,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10416,"num_write_kb":126678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'11296","reported_seq":11343,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.754573+0000","last_change":"2024-03-14T21:14:03.029847+0000","last_active":"2024-03-14T22:05:25.754573+0000","last_peered":"2024-03-14T22:05:25.754573+0000","last_clean":"2024-03-14T22:05:25.754573+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T22:05:25.754573+0000","last_undegraded":"2024-03-14T22:05:25.754573+0000","last_fullsized":"2024-03-14T22:05:25.754573+0000","mapping_epoch":19,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2806,"num_write":9622,"num_write_kb":118224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'8637","reported_seq":8674,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.822320+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T22:05:25.822320+0000","last_peered":"2024-03-14T22:05:25.822320+0000","last_clean":"2024-03-14T22:05:25.822320+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T22:05:25.822320+0000","last_undegraded":"2024-03-14T22:05:25.822320+0000","last_fullsized":"2024-03-14T22:05:25.822320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":7051,"num_write_kb":105885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'9053","reported_seq":9089,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.821226+0000","last_change":"2024-03-14T21:14:03.037915+0000","last_active":"2024-03-14T22:05:25.821226+0000","last_peered":"2024-03-14T22:05:25.821226+0000","last_clean":"2024-03-14T22:05:25.821226+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T22:05:25.821226+0000","last_undegraded":"2024-03-14T22:05:25.821226+0000","last_fullsized":"2024-03-14T22:05:25.821226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:49.207884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":7364,"num_write_kb":104642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'10861","reported_seq":10894,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422940+0000","last_change":"2024-03-14T21:14:02.655762+0000","last_active":"2024-03-14T22:05:23.422940+0000","last_peered":"2024-03-14T22:05:23.422940+0000","last_clean":"2024-03-14T22:05:23.422940+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T22:05:23.422940+0000","last_undegraded":"2024-03-14T22:05:23.422940+0000","last_fullsized":"2024-03-14T22:05:23.422940+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10855,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'9539","reported_seq":9579,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.869948+0000","last_change":"2024-03-14T21:14:03.033091+0000","last_active":"2024-03-14T22:05:25.869948+0000","last_peered":"2024-03-14T22:05:25.869948+0000","last_clean":"2024-03-14T22:05:25.869948+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T22:05:25.869948+0000","last_undegraded":"2024-03-14T22:05:25.869948+0000","last_fullsized":"2024-03-14T22:05:25.869948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":7923,"num_write_kb":107722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10915","reported_seq":10948,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.394449+0000","last_change":"2024-03-14T21:14:02.662899+0000","last_active":"2024-03-14T22:05:23.394449+0000","last_peered":"2024-03-14T22:05:23.394449+0000","last_clean":"2024-03-14T22:05:23.394449+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T22:05:23.394449+0000","last_undegraded":"2024-03-14T22:05:23.394449+0000","last_fullsized":"2024-03-14T22:05:23.394449+0000","mapping_epoch":18,"log_start":"23'900","ondisk_log_start":"23'900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":10937,"num_write_kb":77181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'7036","reported_seq":7070,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.422556+0000","last_change":"2024-03-14T21:14:02.660574+0000","last_active":"2024-03-14T22:05:23.422556+0000","last_peered":"2024-03-14T22:05:23.422556+0000","last_clean":"2024-03-14T22:05:23.422556+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T22:05:23.422556+0000","last_undegraded":"2024-03-14T22:05:23.422556+0000","last_fullsized":"2024-03-14T22:05:23.422556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":7036,"log_dups_size":0,"ondisk_log_size":7036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7046,"num_write_kb":58697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'11487","reported_seq":11521,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.842425+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T22:05:25.842425+0000","last_peered":"2024-03-14T22:05:25.842425+0000","last_clean":"2024-03-14T22:05:25.842425+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T22:05:25.842425+0000","last_undegraded":"2024-03-14T22:05:25.842425+0000","last_fullsized":"2024-03-14T22:05:25.842425+0000","mapping_epoch":19,"log_start":"23'1400","ondisk_log_start":"23'1400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:28:36.858610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":9890,"num_write_kb":106254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'11076","reported_seq":11110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:17.352182+0000","last_change":"2024-03-14T21:14:02.661303+0000","last_active":"2024-03-14T22:05:17.352182+0000","last_peered":"2024-03-14T22:05:17.352182+0000","last_clean":"2024-03-14T22:05:17.352182+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T22:05:17.352182+0000","last_undegraded":"2024-03-14T22:05:17.352182+0000","last_fullsized":"2024-03-14T22:05:17.352182+0000","mapping_epoch":18,"log_start":"23'1000","ondisk_log_start":"23'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11078,"num_write_kb":81064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'10475","reported_seq":10520,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.817331+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T22:05:25.817331+0000","last_peered":"2024-03-14T22:05:25.817331+0000","last_clean":"2024-03-14T22:05:25.817331+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T22:05:25.817331+0000","last_undegraded":"2024-03-14T22:05:25.817331+0000","last_fullsized":"2024-03-14T22:05:25.817331+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2800,"num_write":8909,"num_write_kb":114509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8286","reported_seq":8925,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.553161+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T22:05:15.553161+0000","last_peered":"2024-03-14T22:05:15.553161+0000","last_clean":"2024-03-14T22:05:15.553161+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T22:05:15.553161+0000","last_undegraded":"2024-03-14T22:05:15.553161+0000","last_fullsized":"2024-03-14T22:05:15.553161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8286,"log_dups_size":0,"ondisk_log_size":8286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":607,"num_read_kb":4494,"num_write":8292,"num_write_kb":66794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10203","reported_seq":10241,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.753465+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T22:05:25.753465+0000","last_peered":"2024-03-14T22:05:25.753465+0000","last_clean":"2024-03-14T22:05:25.753465+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T22:05:25.753465+0000","last_undegraded":"2024-03-14T22:05:25.753465+0000","last_fullsized":"2024-03-14T22:05:25.753465+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:09:07.511809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8609,"num_write_kb":104201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'10690","reported_seq":10724,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.418804+0000","last_change":"2024-03-14T21:14:02.660384+0000","last_active":"2024-03-14T22:05:23.418804+0000","last_peered":"2024-03-14T22:05:23.418804+0000","last_clean":"2024-03-14T22:05:23.418804+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T22:05:23.418804+0000","last_undegraded":"2024-03-14T22:05:23.418804+0000","last_fullsized":"2024-03-14T22:05:23.418804+0000","mapping_epoch":18,"log_start":"23'600","ondisk_log_start":"23'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":92935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'9099","reported_seq":9137,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.773805+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T22:05:25.773805+0000","last_peered":"2024-03-14T22:05:25.773805+0000","last_clean":"2024-03-14T22:05:25.773805+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T22:05:25.773805+0000","last_undegraded":"2024-03-14T22:05:25.773805+0000","last_fullsized":"2024-03-14T22:05:25.773805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9099,"log_dups_size":0,"ondisk_log_size":9099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7455,"num_write_kb":104328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9149","reported_seq":9184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:26.341527+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T22:05:26.341527+0000","last_peered":"2024-03-14T22:05:26.341527+0000","last_clean":"2024-03-14T22:05:26.341527+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T22:05:26.341527+0000","last_undegraded":"2024-03-14T22:05:26.341527+0000","last_fullsized":"2024-03-14T22:05:26.341527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9149,"log_dups_size":0,"ondisk_log_size":9149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9159,"num_write_kb":60302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11329","reported_seq":11372,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.925276+0000","last_change":"2024-03-14T21:14:03.032533+0000","last_active":"2024-03-14T22:05:25.925276+0000","last_peered":"2024-03-14T22:05:25.925276+0000","last_clean":"2024-03-14T22:05:25.925276+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T22:05:25.925276+0000","last_undegraded":"2024-03-14T22:05:25.925276+0000","last_fullsized":"2024-03-14T22:05:25.925276+0000","mapping_epoch":19,"log_start":"23'1300","ondisk_log_start":"23'1300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:41.581386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2542,"num_write":9699,"num_write_kb":113170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'13351","reported_seq":13939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.410668+0000","last_change":"2024-03-14T21:14:02.659028+0000","last_active":"2024-03-14T22:05:23.410668+0000","last_peered":"2024-03-14T22:05:23.410668+0000","last_clean":"2024-03-14T22:05:23.410668+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T22:05:23.410668+0000","last_undegraded":"2024-03-14T22:05:23.410668+0000","last_fullsized":"2024-03-14T22:05:23.410668+0000","mapping_epoch":18,"log_start":"23'3300","ondisk_log_start":"23'3300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":557,"num_read_kb":4455,"num_write":13349,"num_write_kb":107014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'9106","reported_seq":9152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.802213+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T22:05:25.802213+0000","last_peered":"2024-03-14T22:05:25.802213+0000","last_clean":"2024-03-14T22:05:25.802213+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T22:05:25.802213+0000","last_undegraded":"2024-03-14T22:05:25.802213+0000","last_fullsized":"2024-03-14T22:05:25.802213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9106,"log_dups_size":0,"ondisk_log_size":9106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3318,"num_write":7536,"num_write_kb":95248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'8009","reported_seq":8045,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:29.211184+0000","last_change":"2024-03-14T21:14:02.663139+0000","last_active":"2024-03-14T22:05:29.211184+0000","last_peered":"2024-03-14T22:05:29.211184+0000","last_clean":"2024-03-14T22:05:29.211184+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T22:05:29.211184+0000","last_undegraded":"2024-03-14T22:05:29.211184+0000","last_fullsized":"2024-03-14T22:05:29.211184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8009,"log_dups_size":0,"ondisk_log_size":8009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8008,"num_write_kb":62700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'9649","reported_seq":9689,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.805938+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T22:05:25.805938+0000","last_peered":"2024-03-14T22:05:25.805938+0000","last_clean":"2024-03-14T22:05:25.805938+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T22:05:25.805938+0000","last_undegraded":"2024-03-14T22:05:25.805938+0000","last_fullsized":"2024-03-14T22:05:25.805938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9649,"log_dups_size":0,"ondisk_log_size":9649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8033,"num_write_kb":100651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5756","reported_seq":5791,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.375166+0000","last_change":"2024-03-14T21:14:02.659773+0000","last_active":"2024-03-14T22:05:23.375166+0000","last_peered":"2024-03-14T22:05:23.375166+0000","last_clean":"2024-03-14T22:05:23.375166+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T22:05:23.375166+0000","last_undegraded":"2024-03-14T22:05:23.375166+0000","last_fullsized":"2024-03-14T22:05:23.375166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5756,"log_dups_size":0,"ondisk_log_size":5756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5754,"num_write_kb":48600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'9764","reported_seq":9800,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.760965+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T22:05:25.760965+0000","last_peered":"2024-03-14T22:05:25.760965+0000","last_clean":"2024-03-14T22:05:25.760965+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T22:05:25.760965+0000","last_undegraded":"2024-03-14T22:05:25.760965+0000","last_fullsized":"2024-03-14T22:05:25.760965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9764,"log_dups_size":0,"ondisk_log_size":9764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:59:58.159496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8135,"num_write_kb":106503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5851","reported_seq":5886,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.412820+0000","last_change":"2024-03-14T21:14:02.663585+0000","last_active":"2024-03-14T22:05:23.412820+0000","last_peered":"2024-03-14T22:05:23.412820+0000","last_clean":"2024-03-14T22:05:23.412820+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T22:05:23.412820+0000","last_undegraded":"2024-03-14T22:05:23.412820+0000","last_fullsized":"2024-03-14T22:05:23.412820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5851,"log_dups_size":0,"ondisk_log_size":5851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":52920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'9086","reported_seq":9130,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.804166+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T22:05:25.804166+0000","last_peered":"2024-03-14T22:05:25.804166+0000","last_clean":"2024-03-14T22:05:25.804166+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T22:05:25.804166+0000","last_undegraded":"2024-03-14T22:05:25.804166+0000","last_fullsized":"2024-03-14T22:05:25.804166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9086,"log_dups_size":0,"ondisk_log_size":9086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:40:05.161282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2548,"num_write":7495,"num_write_kb":101746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8327","reported_seq":8362,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:26.336447+0000","last_change":"2024-03-14T21:14:02.654931+0000","last_active":"2024-03-14T22:05:26.336447+0000","last_peered":"2024-03-14T22:05:26.336447+0000","last_clean":"2024-03-14T22:05:26.336447+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T22:05:26.336447+0000","last_undegraded":"2024-03-14T22:05:26.336447+0000","last_fullsized":"2024-03-14T22:05:26.336447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8327,"log_dups_size":0,"ondisk_log_size":8327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:32:50.279959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16724139,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":8340,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'10136","reported_seq":10172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.901810+0000","last_change":"2024-03-14T21:14:44.859568+0000","last_active":"2024-03-14T22:05:25.901810+0000","last_peered":"2024-03-14T22:05:25.901810+0000","last_clean":"2024-03-14T22:05:25.901810+0000","last_became_active":"2024-03-14T21:14:44.859185+0000","last_became_peered":"2024-03-14T21:14:44.859185+0000","last_unstale":"2024-03-14T22:05:25.901810+0000","last_undegraded":"2024-03-14T22:05:25.901810+0000","last_fullsized":"2024-03-14T22:05:25.901810+0000","mapping_epoch":22,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":8491,"num_write_kb":113136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"23'8046","reported_seq":8081,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430049+0000","last_change":"2024-03-14T21:14:02.663747+0000","last_active":"2024-03-14T22:05:23.430049+0000","last_peered":"2024-03-14T22:05:23.430049+0000","last_clean":"2024-03-14T22:05:23.430049+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T22:05:23.430049+0000","last_undegraded":"2024-03-14T22:05:23.430049+0000","last_fullsized":"2024-03-14T22:05:23.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:59:11.091842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8051,"num_write_kb":62569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'9555","reported_seq":9597,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.773728+0000","last_change":"2024-03-14T21:14:03.030867+0000","last_active":"2024-03-14T22:05:25.773728+0000","last_peered":"2024-03-14T22:05:25.773728+0000","last_clean":"2024-03-14T22:05:25.773728+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T22:05:25.773728+0000","last_undegraded":"2024-03-14T22:05:25.773728+0000","last_fullsized":"2024-03-14T22:05:25.773728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9555,"log_dups_size":0,"ondisk_log_size":9555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:00:23.374788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2042,"num_write":7926,"num_write_kb":109666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'10889","reported_seq":10924,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.438118+0000","last_change":"2024-03-14T21:14:02.659095+0000","last_active":"2024-03-14T22:05:23.438118+0000","last_peered":"2024-03-14T22:05:23.438118+0000","last_clean":"2024-03-14T22:05:23.438118+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T22:05:23.438118+0000","last_undegraded":"2024-03-14T22:05:23.438118+0000","last_fullsized":"2024-03-14T22:05:23.438118+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10896,"num_write_kb":88926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9015","reported_seq":9054,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.831200+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T22:05:25.831200+0000","last_peered":"2024-03-14T22:05:25.831200+0000","last_clean":"2024-03-14T22:05:25.831200+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T22:05:25.831200+0000","last_undegraded":"2024-03-14T22:05:25.831200+0000","last_fullsized":"2024-03-14T22:05:25.831200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9015,"log_dups_size":0,"ondisk_log_size":9015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":7443,"num_write_kb":93234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'11498","reported_seq":11534,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.419685+0000","last_change":"2024-03-14T21:14:02.663441+0000","last_active":"2024-03-14T22:05:23.419685+0000","last_peered":"2024-03-14T22:05:23.419685+0000","last_clean":"2024-03-14T22:05:23.419685+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T22:05:23.419685+0000","last_undegraded":"2024-03-14T22:05:23.419685+0000","last_fullsized":"2024-03-14T22:05:23.419685+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11487,"num_write_kb":85212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'9516","reported_seq":9556,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.796615+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T22:05:25.796615+0000","last_peered":"2024-03-14T22:05:25.796615+0000","last_clean":"2024-03-14T22:05:25.796615+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T22:05:25.796615+0000","last_undegraded":"2024-03-14T22:05:25.796615+0000","last_fullsized":"2024-03-14T22:05:25.796615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9516,"log_dups_size":0,"ondisk_log_size":9516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:03.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":7940,"num_write_kb":115635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8904","reported_seq":8939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421547+0000","last_change":"2024-03-14T21:14:02.653919+0000","last_active":"2024-03-14T22:05:23.421547+0000","last_peered":"2024-03-14T22:05:23.421547+0000","last_clean":"2024-03-14T22:05:23.421547+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T22:05:23.421547+0000","last_undegraded":"2024-03-14T22:05:23.421547+0000","last_fullsized":"2024-03-14T22:05:23.421547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8904,"log_dups_size":0,"ondisk_log_size":8904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:11:12.799763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8892,"num_write_kb":67382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'9427","reported_seq":9465,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.848217+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T22:05:25.848217+0000","last_peered":"2024-03-14T22:05:25.848217+0000","last_clean":"2024-03-14T22:05:25.848217+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T22:05:25.848217+0000","last_undegraded":"2024-03-14T22:05:25.848217+0000","last_fullsized":"2024-03-14T22:05:25.848217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9427,"log_dups_size":0,"ondisk_log_size":9427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:16:14.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7820,"num_write_kb":110543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:02:10.082485+0000","last_change":"2024-03-14T21:13:52.635762+0000","last_active":"2024-03-14T22:02:10.082485+0000","last_peered":"2024-03-14T22:02:10.082485+0000","last_clean":"2024-03-14T22:02:10.082485+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T22:02:10.082485+0000","last_undegraded":"2024-03-14T22:02:10.082485+0000","last_fullsized":"2024-03-14T22:02:10.082485+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:13:51.598575+0000","last_clean_scrub_stamp":"2024-03-14T21:13:51.598575+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:16:29.287152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'8577","reported_seq":8614,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430551+0000","last_change":"2024-03-14T21:14:02.652881+0000","last_active":"2024-03-14T22:05:23.430551+0000","last_peered":"2024-03-14T22:05:23.430551+0000","last_clean":"2024-03-14T22:05:23.430551+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T22:05:23.430551+0000","last_undegraded":"2024-03-14T22:05:23.430551+0000","last_fullsized":"2024-03-14T22:05:23.430551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8577,"log_dups_size":0,"ondisk_log_size":8577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:06:00.476030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":67724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'11133","reported_seq":11172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.818755+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T22:05:25.818755+0000","last_peered":"2024-03-14T22:05:25.818755+0000","last_clean":"2024-03-14T22:05:25.818755+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T22:05:25.818755+0000","last_undegraded":"2024-03-14T22:05:25.818755+0000","last_fullsized":"2024-03-14T22:05:25.818755+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":9507,"num_write_kb":117178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'8676","reported_seq":8714,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.421927+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T22:05:23.421927+0000","last_peered":"2024-03-14T22:05:23.421927+0000","last_clean":"2024-03-14T22:05:23.421927+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T22:05:23.421927+0000","last_undegraded":"2024-03-14T22:05:23.421927+0000","last_fullsized":"2024-03-14T22:05:23.421927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8676,"log_dups_size":0,"ondisk_log_size":8676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:43:46.126855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8672,"num_write_kb":67194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10246","reported_seq":10285,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.855112+0000","last_change":"2024-03-14T21:14:03.033195+0000","last_active":"2024-03-14T22:05:25.855112+0000","last_peered":"2024-03-14T22:05:25.855112+0000","last_clean":"2024-03-14T22:05:25.855112+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T22:05:25.855112+0000","last_undegraded":"2024-03-14T22:05:25.855112+0000","last_fullsized":"2024-03-14T22:05:25.855112+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8656,"num_write_kb":115905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11303","reported_seq":12007,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.404838+0000","last_change":"2024-03-14T21:14:02.658709+0000","last_active":"2024-03-14T22:05:23.404838+0000","last_peered":"2024-03-14T22:05:23.404838+0000","last_clean":"2024-03-14T22:05:23.404838+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T22:05:23.404838+0000","last_undegraded":"2024-03-14T22:05:23.404838+0000","last_fullsized":"2024-03-14T22:05:23.404838+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:40:03.029687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":669,"num_read_kb":4516,"num_write":11296,"num_write_kb":89038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'9443","reported_seq":9484,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.783718+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T22:05:25.783718+0000","last_peered":"2024-03-14T22:05:25.783718+0000","last_clean":"2024-03-14T22:05:25.783718+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T22:05:25.783718+0000","last_undegraded":"2024-03-14T22:05:25.783718+0000","last_fullsized":"2024-03-14T22:05:25.783718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9443,"log_dups_size":0,"ondisk_log_size":9443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7837,"num_write_kb":123513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'9296","reported_seq":9333,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.408089+0000","last_change":"2024-03-14T21:14:02.657401+0000","last_active":"2024-03-14T22:05:23.408089+0000","last_peered":"2024-03-14T22:05:23.408089+0000","last_clean":"2024-03-14T22:05:23.408089+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T22:05:23.408089+0000","last_undegraded":"2024-03-14T22:05:23.408089+0000","last_fullsized":"2024-03-14T22:05:23.408089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9294,"num_write_kb":71830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'9546","reported_seq":9583,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.758775+0000","last_change":"2024-03-14T21:14:03.036152+0000","last_active":"2024-03-14T22:05:25.758775+0000","last_peered":"2024-03-14T22:05:25.758775+0000","last_clean":"2024-03-14T22:05:25.758775+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T22:05:25.758775+0000","last_undegraded":"2024-03-14T22:05:25.758775+0000","last_fullsized":"2024-03-14T22:05:25.758775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:14.794102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":7936,"num_write_kb":109238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'11134","reported_seq":11172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.437168+0000","last_change":"2024-03-14T21:14:02.653065+0000","last_active":"2024-03-14T22:05:23.437168+0000","last_peered":"2024-03-14T22:05:23.437168+0000","last_clean":"2024-03-14T22:05:23.437168+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T22:05:23.437168+0000","last_undegraded":"2024-03-14T22:05:23.437168+0000","last_fullsized":"2024-03-14T22:05:23.437168+0000","mapping_epoch":18,"log_start":"23'1100","ondisk_log_start":"23'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11130,"num_write_kb":85841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'12171","reported_seq":12207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.885446+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T22:05:25.885446+0000","last_peered":"2024-03-14T22:05:25.885446+0000","last_clean":"2024-03-14T22:05:25.885446+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T22:05:25.885446+0000","last_undegraded":"2024-03-14T22:05:25.885446+0000","last_fullsized":"2024-03-14T22:05:25.885446+0000","mapping_epoch":19,"log_start":"23'2100","ondisk_log_start":"23'2100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:33:36.762286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10560,"num_write_kb":124816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'9283","reported_seq":9320,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.431083+0000","last_change":"2024-03-14T21:14:02.659612+0000","last_active":"2024-03-14T22:05:23.431083+0000","last_peered":"2024-03-14T22:05:23.431083+0000","last_clean":"2024-03-14T22:05:23.431083+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T22:05:23.431083+0000","last_undegraded":"2024-03-14T22:05:23.431083+0000","last_fullsized":"2024-03-14T22:05:23.431083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9283,"log_dups_size":0,"ondisk_log_size":9283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:22.505692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9294,"num_write_kb":71728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9772","reported_seq":9814,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.730809+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T22:05:25.730809+0000","last_peered":"2024-03-14T22:05:25.730809+0000","last_clean":"2024-03-14T22:05:25.730809+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T22:05:25.730809+0000","last_undegraded":"2024-03-14T22:05:25.730809+0000","last_fullsized":"2024-03-14T22:05:25.730809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9772,"log_dups_size":0,"ondisk_log_size":9772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8091,"num_write_kb":107879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'8359","reported_seq":8989,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:26.740982+0000","last_change":"2024-03-14T21:14:02.662122+0000","last_active":"2024-03-14T22:05:26.740982+0000","last_peered":"2024-03-14T22:05:26.740982+0000","last_clean":"2024-03-14T22:05:26.740982+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T22:05:26.740982+0000","last_undegraded":"2024-03-14T22:05:26.740982+0000","last_fullsized":"2024-03-14T22:05:26.740982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8359,"log_dups_size":0,"ondisk_log_size":8359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":593,"num_read_kb":4491,"num_write":9534,"num_write_kb":73072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'10081","reported_seq":10125,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.868288+0000","last_change":"2024-03-14T21:14:03.037921+0000","last_active":"2024-03-14T22:05:25.868288+0000","last_peered":"2024-03-14T22:05:25.868288+0000","last_clean":"2024-03-14T22:05:25.868288+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T22:05:25.868288+0000","last_undegraded":"2024-03-14T22:05:25.868288+0000","last_fullsized":"2024-03-14T22:05:25.868288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:47:45.911439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8532,"num_write_kb":114847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5880","reported_seq":5916,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.420296+0000","last_change":"2024-03-14T21:14:02.665745+0000","last_active":"2024-03-14T22:05:23.420296+0000","last_peered":"2024-03-14T22:05:23.420296+0000","last_clean":"2024-03-14T22:05:23.420296+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T22:05:23.420296+0000","last_undegraded":"2024-03-14T22:05:23.420296+0000","last_fullsized":"2024-03-14T22:05:23.420296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":5880,"log_dups_size":0,"ondisk_log_size":5880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5885,"num_write_kb":44672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'10915","reported_seq":10958,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.760889+0000","last_change":"2024-03-14T21:14:03.029269+0000","last_active":"2024-03-14T22:05:25.760889+0000","last_peered":"2024-03-14T22:05:25.760889+0000","last_clean":"2024-03-14T22:05:25.760889+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T22:05:25.760889+0000","last_undegraded":"2024-03-14T22:05:25.760889+0000","last_fullsized":"2024-03-14T22:05:25.760889+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":9274,"num_write_kb":112928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'4360","reported_seq":4396,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:15.551031+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T22:05:15.551031+0000","last_peered":"2024-03-14T22:05:15.551031+0000","last_clean":"2024-03-14T22:05:15.551031+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T22:05:15.551031+0000","last_undegraded":"2024-03-14T22:05:15.551031+0000","last_fullsized":"2024-03-14T22:05:15.551031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:48:40.576562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4362,"num_write_kb":43693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'9887","reported_seq":9935,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.901640+0000","last_change":"2024-03-14T21:14:03.039549+0000","last_active":"2024-03-14T22:05:25.901640+0000","last_peered":"2024-03-14T22:05:25.901640+0000","last_clean":"2024-03-14T22:05:25.901640+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T22:05:25.901640+0000","last_undegraded":"2024-03-14T22:05:25.901640+0000","last_fullsized":"2024-03-14T22:05:25.901640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9887,"log_dups_size":0,"ondisk_log_size":9887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:23.510516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":8266,"num_write_kb":116331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'6406","reported_seq":6443,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.426775+0000","last_change":"2024-03-14T21:14:02.660292+0000","last_active":"2024-03-14T22:05:23.426775+0000","last_peered":"2024-03-14T22:05:23.426775+0000","last_clean":"2024-03-14T22:05:23.426775+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T22:05:23.426775+0000","last_undegraded":"2024-03-14T22:05:23.426775+0000","last_fullsized":"2024-03-14T22:05:23.426775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6406,"log_dups_size":0,"ondisk_log_size":6406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6417,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'10927","reported_seq":10962,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.842283+0000","last_change":"2024-03-14T21:14:03.040817+0000","last_active":"2024-03-14T22:05:25.842283+0000","last_peered":"2024-03-14T22:05:25.842283+0000","last_clean":"2024-03-14T22:05:25.842283+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T22:05:25.842283+0000","last_undegraded":"2024-03-14T22:05:25.842283+0000","last_fullsized":"2024-03-14T22:05:25.842283+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:32:12.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":120976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'9352","reported_seq":9389,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.434775+0000","last_change":"2024-03-14T21:14:02.661073+0000","last_active":"2024-03-14T22:05:23.434775+0000","last_peered":"2024-03-14T22:05:23.434775+0000","last_clean":"2024-03-14T22:05:23.434775+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T22:05:23.434775+0000","last_undegraded":"2024-03-14T22:05:23.434775+0000","last_fullsized":"2024-03-14T22:05:23.434775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9352,"log_dups_size":0,"ondisk_log_size":9352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9374,"num_write_kb":71713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'10198","reported_seq":10236,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.810182+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T22:05:25.810182+0000","last_peered":"2024-03-14T22:05:25.810182+0000","last_clean":"2024-03-14T22:05:25.810182+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T22:05:25.810182+0000","last_undegraded":"2024-03-14T22:05:25.810182+0000","last_fullsized":"2024-03-14T22:05:25.810182+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:42:42.849879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8519,"num_write_kb":106953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'11401","reported_seq":11837,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:26.332015+0000","last_change":"2024-03-14T21:14:02.663131+0000","last_active":"2024-03-14T22:05:26.332015+0000","last_peered":"2024-03-14T22:05:26.332015+0000","last_clean":"2024-03-14T22:05:26.332015+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T22:05:26.332015+0000","last_undegraded":"2024-03-14T22:05:26.332015+0000","last_fullsized":"2024-03-14T22:05:26.332015+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:12:32.783095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13689588,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":402,"num_read_kb":1363,"num_write":11407,"num_write_kb":86544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'13463","reported_seq":13504,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.773498+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T22:05:25.773498+0000","last_peered":"2024-03-14T22:05:25.773498+0000","last_clean":"2024-03-14T22:05:25.773498+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T22:05:25.773498+0000","last_undegraded":"2024-03-14T22:05:25.773498+0000","last_fullsized":"2024-03-14T22:05:25.773498+0000","mapping_epoch":19,"log_start":"23'3400","ondisk_log_start":"23'3400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T07:54:47.525696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":11847,"num_write_kb":149794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'10677","reported_seq":10710,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.871083+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T22:05:25.871083+0000","last_peered":"2024-03-14T22:05:25.871083+0000","last_clean":"2024-03-14T22:05:25.871083+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T22:05:25.871083+0000","last_undegraded":"2024-03-14T22:05:25.871083+0000","last_fullsized":"2024-03-14T22:05:25.871083+0000","mapping_epoch":19,"log_start":"23'600","ondisk_log_start":"23'600","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:28.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":8986,"num_write_kb":111494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6864","reported_seq":6897,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.409531+0000","last_change":"2024-03-14T21:14:02.660393+0000","last_active":"2024-03-14T22:05:23.409531+0000","last_peered":"2024-03-14T22:05:23.409531+0000","last_clean":"2024-03-14T22:05:23.409531+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T22:05:23.409531+0000","last_undegraded":"2024-03-14T22:05:23.409531+0000","last_fullsized":"2024-03-14T22:05:23.409531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6864,"log_dups_size":0,"ondisk_log_size":6864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:15:13.789278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6885,"num_write_kb":66562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'11249","reported_seq":11286,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.846608+0000","last_change":"2024-03-14T21:14:44.859446+0000","last_active":"2024-03-14T22:05:25.846608+0000","last_peered":"2024-03-14T22:05:25.846608+0000","last_clean":"2024-03-14T22:05:25.846608+0000","last_became_active":"2024-03-14T21:14:44.859128+0000","last_became_peered":"2024-03-14T21:14:44.859128+0000","last_unstale":"2024-03-14T22:05:25.846608+0000","last_undegraded":"2024-03-14T22:05:25.846608+0000","last_fullsized":"2024-03-14T22:05:25.846608+0000","mapping_epoch":22,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:32:11.377675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1780,"num_write":9572,"num_write_kb":116659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15422","reported_seq":16524,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.386679+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T22:05:23.386679+0000","last_peered":"2024-03-14T22:05:23.386679+0000","last_clean":"2024-03-14T22:05:23.386679+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T22:05:23.386679+0000","last_undegraded":"2024-03-14T22:05:23.386679+0000","last_fullsized":"2024-03-14T22:05:23.386679+0000","mapping_epoch":18,"log_start":"23'5400","ondisk_log_start":"23'5400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:39:48.203059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1069,"num_read_kb":4840,"num_write":15404,"num_write_kb":112501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'10432","reported_seq":10480,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.818270+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T22:05:25.818270+0000","last_peered":"2024-03-14T22:05:25.818270+0000","last_clean":"2024-03-14T22:05:25.818270+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T22:05:25.818270+0000","last_undegraded":"2024-03-14T22:05:25.818270+0000","last_fullsized":"2024-03-14T22:05:25.818270+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2796,"num_write":8857,"num_write_kb":109660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'8706","reported_seq":8744,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.407645+0000","last_change":"2024-03-14T21:14:02.654963+0000","last_active":"2024-03-14T22:05:23.407645+0000","last_peered":"2024-03-14T22:05:23.407645+0000","last_clean":"2024-03-14T22:05:23.407645+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T22:05:23.407645+0000","last_undegraded":"2024-03-14T22:05:23.407645+0000","last_fullsized":"2024-03-14T22:05:23.407645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8706,"log_dups_size":0,"ondisk_log_size":8706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:40:06.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8698,"num_write_kb":67327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'11837","reported_seq":11875,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.812840+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T22:05:25.812840+0000","last_peered":"2024-03-14T22:05:25.812840+0000","last_clean":"2024-03-14T22:05:25.812840+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T22:05:25.812840+0000","last_undegraded":"2024-03-14T22:05:25.812840+0000","last_fullsized":"2024-03-14T22:05:25.812840+0000","mapping_epoch":19,"log_start":"23'1800","ondisk_log_start":"23'1800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:14:48.701662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":10179,"num_write_kb":107836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6046","reported_seq":6084,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.413066+0000","last_change":"2024-03-14T21:14:02.656125+0000","last_active":"2024-03-14T22:05:23.413066+0000","last_peered":"2024-03-14T22:05:23.413066+0000","last_clean":"2024-03-14T22:05:23.413066+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T22:05:23.413066+0000","last_undegraded":"2024-03-14T22:05:23.413066+0000","last_fullsized":"2024-03-14T22:05:23.413066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6048,"num_write_kb":48854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":345,"num_read_kb":87814,"num_write":552757,"num_write_kb":7123402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624278,"ondisk_log_size":624278,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":256907613,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":3941,"num_read_kb":24159,"num_write":290960,"num_write_kb":2282127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":513884160,"data_stored":513815226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271709,"ondisk_log_size":271709,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575472,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252664,"kb_used_data":112496,"kb_used_omap":1,"kb_used_meta":1140158,"kb_avail":93119176,"statfs":{"total":96636764160,"available":95354036224,"internally_reserved":0,"allocated":115195904,"data_stored":115031593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1167522059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45500000000000002,"15min":0.48699999999999999},"min":{"1min":0.26100000000000001,"5min":0.23000000000000001,"15min":0.19500000000000001},"max":{"1min":0.57499999999999996,"5min":2.782,"15min":4.7999999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":0.63900000000000001,"5min":2.8519999999999999,"15min":3.9239999999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":0.66000000000000003,"5min":1.8520000000000001,"15min":3.903},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.51100000000000001,"15min":0.55900000000000005},"min":{"1min":0.30099999999999999,"5min":0.28599999999999998,"15min":0.26400000000000001},"max":{"1min":0.60599999999999998,"5min":6.1440000000000001,"15min":6.6689999999999996},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.30199999999999999,"5min":0.28100000000000003,"15min":0.25900000000000001},"max":{"1min":0.63500000000000001,"5min":3.8210000000000002,"15min":4.8310000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.59199999999999997,"15min":0.54700000000000004},"min":{"1min":0.32000000000000001,"5min":0.29799999999999999,"15min":0.25900000000000001},"max":{"1min":0.59799999999999998,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67200000000000004,"5min":6.1820000000000004,"15min":6.6909999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.5,"15min":0.52500000000000002},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.622,"5min":1.9339999999999999,"15min":4.851},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.45900000000000002},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.68700000000000006,"5min":1.226,"15min":6.7220000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40899999999999997,"15min":0.44700000000000001},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.23300000000000001},"max":{"1min":0.69899999999999995,"5min":1.492,"15min":5.2679999999999998},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42999999999999999,"15min":0.44500000000000001},"min":{"1min":0.313,"5min":0.255,"15min":0.24199999999999999},"max":{"1min":0.71799999999999997,"5min":1.3089999999999999,"15min":3.8740000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46600000000000003,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.755,"5min":1.321,"15min":6.6459999999999999},"last":0.63}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.49399999999999999},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69599999999999995,"5min":1.2529999999999999,"15min":4.157},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48499999999999999,"15min":0.55800000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":1.2509999999999999,"15min":6.6109999999999998},"last":0.73699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575471,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101820,"kb_used_data":69876,"kb_used_omap":1,"kb_used_meta":1031934,"kb_avail":93270020,"statfs":{"total":96636764160,"available":95508500480,"internally_reserved":0,"allocated":71553024,"data_stored":71391599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1056700683},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.47299999999999998,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.188},"max":{"1min":2.899,"5min":2.899,"15min":17.885999999999999},"last":0.502},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.23000000000000001},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.71799999999999997}]},{"osd":1,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.52000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.216},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":7.226},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":1.224,"15min":7.1699999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.6189999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.26800000000000002},"max":{"1min":3.282,"5min":3.282,"15min":7.2489999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":1.2470000000000001,"15min":7.1470000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.61799999999999999,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.29899999999999999},"max":{"1min":6.1660000000000004,"5min":6.1660000000000004,"15min":6.1660000000000004},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.468,"5min":2.468,"15min":7.3330000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.49199999999999999,"15min":0.45300000000000001},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":6.0229999999999997,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":8.234,"5min":8.234,"15min":8.234},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48799999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.90600000000000003,"5min":1.3340000000000001,"15min":7.5380000000000003},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.54100000000000004,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.752},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":7.2039999999999997},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575471,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1151736,"kb_used_data":69872,"kb_used_omap":1,"kb_used_meta":1081854,"kb_avail":93220104,"statfs":{"total":96636764160,"available":95457386496,"internally_reserved":0,"allocated":71548928,"data_stored":71391017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1107818763},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51500000000000001,"15min":0.55500000000000005},"min":{"1min":0.27300000000000002,"5min":0.221,"15min":0.192},"max":{"1min":0.71199999999999997,"5min":3.2149999999999999,"15min":7.0990000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52900000000000003,"15min":0.57499999999999996},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.21099999999999999},"max":{"1min":0.66800000000000004,"5min":5.7359999999999998,"15min":12.281000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.68100000000000005,"15min":0.60399999999999998},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.20599999999999999},"max":{"1min":0.70299999999999996,"5min":14.952,"15min":14.952},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49299999999999999,"15min":0.54500000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.222},"max":{"1min":0.63600000000000001,"5min":3.1339999999999999,"15min":8.8539999999999992},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.75800000000000001,"15min":0.63100000000000001},"min":{"1min":0.28699999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":0.70599999999999996,"5min":18.623000000000001,"15min":18.623000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.96399999999999997,"15min":0.70699999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.67000000000000004,"5min":30.004999999999999,"15min":30.004999999999999},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.25},"max":{"1min":0.66200000000000003,"5min":3.6600000000000001,"15min":9.2309999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.57699999999999996},"min":{"1min":0.34000000000000002,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":0.82699999999999996,"15min":7.3780000000000001},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.59899999999999998,"15min":0.53700000000000003},"min":{"1min":0.22700000000000001,"5min":0.219,"15min":0.20999999999999999},"max":{"1min":0.58399999999999996,"5min":12.414,"15min":12.414},"last":0.497},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.496,"15min":0.57099999999999995},"min":{"1min":0.308,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60799999999999998,"5min":3.1970000000000001,"15min":8.9220000000000006},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.44600000000000001,"15min":0.46600000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.22},"max":{"1min":0.55600000000000005,"5min":1.905,"15min":7.4000000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.61599999999999999,"15min":0.58299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.24299999999999999},"max":{"1min":0.65700000000000003,"5min":10.278,"15min":10.278},"last":0.628}]},{"osd":7,"last update":"Thu Mar 14 22:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.52300000000000002,"15min":0.55100000000000005},"min":{"1min":0.27500000000000002,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":0.64200000000000002,"5min":3.9820000000000002,"15min":7.1619999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.28399999999999997},"max":{"1min":0.623,"5min":3.7789999999999999,"15min":7.3310000000000004},"last":0.61099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575472,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":965948,"kb_used_data":42292,"kb_used_omap":1,"kb_used_meta":923646,"kb_avail":93405892,"statfs":{"total":96636764160,"available":95647633408,"internally_reserved":0,"allocated":43307008,"data_stored":43137601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":945813771},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.495,"15min":0.55600000000000005},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":0.73799999999999999,"5min":2.23,"15min":15.013},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.26000000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.72599999999999998,"5min":2.1139999999999999,"15min":5.6879999999999997},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.498,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":1.806,"15min":4.6479999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.52000000000000002},"min":{"1min":0.28599999999999998,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":0.64900000000000002,"5min":0.77800000000000002,"15min":4.343},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.503},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.70699999999999996,"5min":0.97099999999999997,"15min":5.5540000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.57499999999999996,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.33000000000000002,"15min":0.26000000000000001},"max":{"1min":0.75700000000000001,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.625}]},{"osd":3,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48699999999999999,"15min":0.55900000000000005},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.22},"max":{"1min":0.67700000000000005,"5min":2.1320000000000001,"15min":9.9800000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.56499999999999995},"min":{"1min":0.33000000000000002,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":0.77200000000000002,"5min":1.8919999999999999,"15min":5.899},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41499999999999998,"15min":0.40999999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.20599999999999999},"max":{"1min":0.61099999999999999,"5min":0.71599999999999997,"15min":3.0470000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.41899999999999998,"15min":0.45100000000000001},"min":{"1min":0.28100000000000003,"5min":0.26600000000000001,"15min":0.22700000000000001},"max":{"1min":0.61699999999999999,"5min":0.69999999999999996,"15min":4.2460000000000004},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44500000000000001,"15min":0.44600000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.76100000000000001,"5min":0.90900000000000003,"15min":3.8650000000000002},"last":0.62},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45700000000000002,"15min":0.45900000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":1.5680000000000001},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Mar 14 22:04:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.30299999999999999,"15min":0.23400000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":8.7639999999999993},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.27400000000000002},"max":{"1min":0.69399999999999995,"5min":0.94799999999999995,"15min":3.7850000000000001},"last":0.54500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1131324,"kb_used_data":57588,"kb_used_omap":1,"kb_used_meta":1073726,"kb_avail":93240516,"statfs":{"total":96636764160,"available":95478288384,"internally_reserved":0,"allocated":58970112,"data_stored":58808687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1099495691},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.57599999999999996,"15min":0.53000000000000003},"min":{"1min":0.23000000000000001,"5min":0.20499999999999999,"15min":0.17000000000000001},"max":{"1min":0.52300000000000002,"5min":7.7270000000000003,"15min":14.695},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.23799999999999999,"5min":0.19900000000000001,"15min":0.186},"max":{"1min":0.51900000000000002,"5min":7.9770000000000003,"15min":8.9649999999999999},"last":0.377}]},{"osd":1,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.378,"15min":0.45500000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":0.71799999999999997,"5min":0.93500000000000005,"15min":5.5289999999999999},"last":0.435},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.53700000000000003,"15min":0.60699999999999998},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.20100000000000001},"max":{"1min":0.83899999999999997,"5min":6.1740000000000004,"15min":14.898},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.61599999999999999,"15min":0.57099999999999995},"min":{"1min":0.251,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.83099999999999996,"5min":9.4079999999999995,"15min":9.5150000000000006},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.55900000000000005},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.23100000000000001},"max":{"1min":0.96199999999999997,"5min":2.9430000000000001,"15min":15.263},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.71899999999999997,"15min":0.74399999999999999},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":0.77700000000000002,"5min":9.6080000000000005,"15min":14.071999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.622,"15min":0.69499999999999995},"min":{"1min":0.40200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78300000000000003,"5min":3.5710000000000002,"15min":14.109},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.70899999999999996,"15min":0.68200000000000005},"min":{"1min":0.36499999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":11.443,"15min":11.443},"last":0.626},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.74199999999999999,"15min":0.77000000000000002},"min":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.27800000000000002},"max":{"1min":0.78000000000000003,"5min":7.6399999999999997,"15min":14.550000000000001},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.82999999999999996,"15min":0.76500000000000001},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.29299999999999998},"max":{"1min":0.78400000000000003,"5min":9.7200000000000006,"15min":9.7200000000000006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.78200000000000003,"15min":0.69899999999999995},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.35599999999999998},"max":{"1min":0.81100000000000005,"5min":15.339,"15min":15.339},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Mar 14 22:04:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.81599999999999995,"15min":0.77700000000000002},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":7.7030000000000003,"15min":14.726000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.83399999999999996,"15min":0.76300000000000001},"min":{"1min":0.433,"5min":0.379,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":9.5079999999999991,"15min":9.5079999999999991},"last":0.65800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575472,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649796,"kb_used_data":45244,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":93722044,"statfs":{"total":96636764160,"available":95971373056,"internally_reserved":0,"allocated":46329856,"data_stored":46172026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":619051275},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.61299999999999999,"15min":0.45800000000000002},"min":{"1min":0.24399999999999999,"5min":0.184,"15min":0.182},"max":{"1min":2.347,"5min":10.869,"15min":10.869},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.67800000000000005,"15min":0.55200000000000005},"min":{"1min":0.23799999999999999,"5min":0.20100000000000001,"15min":0.189},"max":{"1min":1.107,"5min":9.6649999999999991,"15min":12.317},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.53100000000000003,"15min":0.45600000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":2.0049999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.20100000000000001},"max":{"1min":0.96799999999999997,"5min":7.8090000000000002,"15min":9.5429999999999993},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.66700000000000004,"15min":0.61899999999999999},"min":{"1min":0.26500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56499999999999995,"5min":13.130000000000001,"15min":27.355},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52800000000000002,"15min":0.57099999999999995},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.26000000000000001},"max":{"1min":1.929,"5min":3.4390000000000001,"15min":12.451000000000001},"last":0.379}]},{"osd":4,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83099999999999996,"15min":0.71999999999999997},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.28899999999999998},"max":{"1min":7.1310000000000002,"5min":8.0440000000000005,"15min":12.478999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.79800000000000004,"15min":0.73699999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.30599999999999999},"max":{"1min":0.86899999999999999,"5min":7.9409999999999998,"15min":12.391999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.0529999999999999,"15min":0.74099999999999999},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":0.97299999999999998,"5min":20.84,"15min":20.84},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69199999999999995,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.0329999999999999,"5min":4.7240000000000002,"15min":11.762},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.69699999999999995,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34300000000000003},"max":{"1min":0.79800000000000004,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.78000000000000003,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.9590000000000001,"5min":13.177,"15min":13.177},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:04:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.79400000000000004,"15min":0.74099999999999999},"min":{"1min":0.42599999999999999,"5min":0.35099999999999998,"15min":0.32600000000000001},"max":{"1min":1.8420000000000001,"5min":6.2309999999999999,"15min":11.865},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834575471,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1031244,"kb_used_data":57540,"kb_used_omap":1,"kb_used_meta":973694,"kb_avail":93340596,"statfs":{"total":96636764160,"available":95580770304,"internally_reserved":0,"allocated":58920960,"data_stored":58755098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":997062923},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49399999999999999,"15min":0.439},"min":{"1min":0.19400000000000001,"5min":0.186,"15min":0.184},"max":{"1min":0.89000000000000001,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.219,"5min":0.219,"15min":0.19700000000000001},"max":{"1min":2.145,"5min":6.5339999999999998,"15min":9.2249999999999996},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54800000000000004,"15min":0.47799999999999998},"min":{"1min":0.23400000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":2.161,"5min":13.553000000000001,"15min":13.553000000000001},"last":0.375},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":2.0910000000000002,"5min":6.6050000000000004,"15min":7.1180000000000003},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.70699999999999996,"15min":0.64000000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.22900000000000001},"max":{"1min":2.1190000000000002,"5min":10.236000000000001,"15min":24.879000000000001},"last":0.435},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.53400000000000003},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.22900000000000001},"max":{"1min":1.4670000000000001,"5min":2.2290000000000001,"15min":20.995000000000001},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":7.1609999999999996,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":8.4420000000000002,"5min":8.4420000000000002,"15min":8.4420000000000002},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.58199999999999996},"min":{"1min":0.34699999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9059999999999999,"5min":3.5800000000000001,"15min":8.6419999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.79100000000000004,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.96699999999999997,"5min":16.943999999999999,"15min":16.943999999999999},"last":0.629}]},{"osd":6,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.0270000000000001,"5min":3.5409999999999999,"15min":7.468},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1,"5min":3.4950000000000001,"15min":5.5030000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:04:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":2.5169999999999999,"5min":8.593,"15min":9.3209999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.40200000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.496,"5min":6.7859999999999996,"15min":10.346},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575472,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":871104,"kb_used_data":49976,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":93500736,"statfs":{"total":96636764160,"available":95744753664,"internally_reserved":0,"allocated":51175424,"data_stored":51009794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":840825099},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.45500000000000002},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.17499999999999999},"max":{"1min":2.8149999999999999,"5min":12.734,"15min":12.734},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52000000000000002,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":3.976,"5min":9.3620000000000001,"15min":9.3620000000000001},"last":0.35799999999999998}]},{"osd":2,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.843,"5min":15.375,"15min":15.375},"last":0.376},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.48099999999999998},"min":{"1min":0.28399999999999997,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.008,"5min":9.2959999999999994,"15min":9.2959999999999994},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.20699999999999999},"max":{"1min":2.7040000000000002,"5min":8.7230000000000008,"15min":8.7230000000000008},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":2.9169999999999998,"5min":10.928000000000001,"15min":12.864000000000001},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0350000000000001,"5min":9.2319999999999993,"15min":9.2319999999999993},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":3.7930000000000001,"15min":7.5499999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.61899999999999999},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":3.0790000000000002,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.68700000000000006},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"max":{"1min":2.8660000000000001,"5min":10.135999999999999,"15min":11.068},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.69699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.8959999999999999,"5min":12.77,"15min":12.77},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":4.0579999999999998,"5min":9.391,"15min":9.391},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73299999999999998,"15min":0.67900000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.0830000000000002,"5min":9.3290000000000006,"15min":9.3290000000000006},"last":1.05},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.71499999999999997,"15min":0.65400000000000003},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.33300000000000002},"max":{"1min":0.78900000000000003,"5min":7.8739999999999997,"15min":7.8739999999999997},"last":0.98699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58679296,"data_stored":58667339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":46088192,"data_stored":46084267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43065344,"data_stored":43049842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":71307264,"data_stored":71303258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":71311360,"data_stored":71303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114360320,"data_stored":114353466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T22:05:32.743 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.744 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.745 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.746 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.747 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.748 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.749 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.749 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.749 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.749 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.749 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.749 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.750 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-14T21:13:51.598575+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=13, tm_sec=51, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.751 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.752 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.753 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.754 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.754 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.754 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.754 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.754 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.754 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:32.754 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-14T22:05:52.755 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T22:05:52.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.875+0000 7fd6742af640 1 Processor -- start 2024-03-14T22:05:52.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.875+0000 7fd6742af640 1 -- start start 2024-03-14T22:05:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd6742af640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd66c0b3a90 0x7fd66c0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd6742af640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3210 0x7fd66c152020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd6742af640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66c071290 con 0x7fd66c141be0 2024-03-14T22:05:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd6742af640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66c06ddd0 con 0x7fd66c0b3a90 2024-03-14T22:05:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd6742af640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0b3680 con 0x7fd66c0b3210 2024-03-14T22:05:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd67204b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd66c141be0 0x7fd66c0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:38684/0 (socket says 172.21.15.49:38684) 2024-03-14T22:05:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd67204b640 1 -- 172.21.15.49:0/3121392786 learned_addr learned my addr 172.21.15.49:0/3121392786 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:05:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.876+0000 7fd67184a640 1 --2- 172.21.15.49:0/3121392786 >> v2:172.21.15.49:3300/0 conn(0x7fd66c0b3a90 0x7fd66c0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.877+0000 7fd67284c640 1 --2- 172.21.15.49:0/3121392786 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3210 0x7fd66c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.877+0000 7fd67184a640 1 -- 172.21.15.49:0/3121392786 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3210 msgr2=0x7fd66c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:05:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.877+0000 7fd67184a640 1 --2- 172.21.15.49:0/3121392786 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3210 0x7fd66c152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.877+0000 7fd67184a640 1 -- 172.21.15.49:0/3121392786 >> v1:172.21.15.49:6789/0 conn(0x7fd66c141be0 legacy=0x7fd66c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.877+0000 7fd67184a640 1 -- 172.21.15.49:0/3121392786 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c070b90 con 0x7fd66c0b3a90 2024-03-14T22:05:52.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.877+0000 7fd67284c640 1 --2- 172.21.15.49:0/3121392786 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3210 0x7fd66c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.877+0000 7fd67184a640 1 --2- 172.21.15.49:0/3121392786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3a90 0x7fd66c0b2cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6600097a0 tx=0x7fd66002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735ed0fd5e5be05d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd671049640 1 -- 172.21.15.49:0/3121392786 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660031830 con 0x7fd66c0b3a90 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd671049640 1 -- 172.21.15.49:0/3121392786 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660031990 con 0x7fd66c0b3a90 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd671049640 1 -- 172.21.15.49:0/3121392786 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660031d20 con 0x7fd66c0b3a90 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 -- 172.21.15.49:0/3121392786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3a90 msgr2=0x7fd66c0b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 --2- 172.21.15.49:0/3121392786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3a90 0x7fd66c0b2cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6600097a0 tx=0x7fd66002b8d0 comp rx=0 tx=0).stop 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 -- 172.21.15.49:0/3121392786 shutdown_connections 2024-03-14T22:05:52.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 --2- 172.21.15.49:0/3121392786 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3210 0x7fd66c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:52.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 --2- 172.21.15.49:0/3121392786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3a90 0x7fd66c0b2cd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:52.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 -- 172.21.15.49:0/3121392786 >> 172.21.15.49:0/3121392786 conn(0x7fd66c0b9c30 msgr2=0x7fd66c0ba040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:52.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 -- 172.21.15.49:0/3121392786 shutdown_connections 2024-03-14T22:05:52.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.878+0000 7fd6742af640 1 -- 172.21.15.49:0/3121392786 wait complete. 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.879+0000 7fd6742af640 1 Processor -- start 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.879+0000 7fd6742af640 1 -- start start 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd6742af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3210 0x7fd66c1e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd6742af640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3a90 0x7fd66c1d2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd6742af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd66c141be0 0x7fd66c1d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd6742af640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c1e4210 con 0x7fd66c0b3210 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd6742af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c1e3a70 con 0x7fd66c141be0 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd6742af640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c146060 con 0x7fd66c0b3a90 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67204b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3210 0x7fd66c1e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67204b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3210 0x7fd66c1e3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58354/0 (socket says 172.21.15.49:58354) 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67204b640 1 -- 172.21.15.49:0/1065079819 learned_addr learned my addr 172.21.15.49:0/1065079819 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67184a640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3a90 0x7fd66c1d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67284c640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd66c141be0 0x7fd66c1d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67204b640 1 -- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3a90 msgr2=0x7fd66c1d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:52.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67204b640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3a90 0x7fd66c1d2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67204b640 1 -- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd66c141be0 msgr2=0x7fd66c1d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.880+0000 7fd67204b640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd66c141be0 0x7fd66c1d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.881+0000 7fd67204b640 1 -- 172.21.15.49:0/1065079819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009400 con 0x7fd66c0b3210 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.881+0000 7fd67204b640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3210 0x7fd66c1e3530 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c4a0 tx=0x7fd66400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.881+0000 7fd66affd640 1 -- 172.21.15.49:0/1065079819 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664005ce0 con 0x7fd66c0b3210 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.881+0000 7fd66affd640 1 -- 172.21.15.49:0/1065079819 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd664005e40 con 0x7fd66c0b3210 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.881+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66c1e3f30 con 0x7fd66c0b3210 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.881+0000 7fd66affd640 1 -- 172.21.15.49:0/1065079819 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664007270 con 0x7fd66c0b3210 2024-03-14T22:05:52.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.881+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c0b81e0 con 0x7fd66c0b3210 2024-03-14T22:05:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.882+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66c073540 con 0x7fd66c0b3210 2024-03-14T22:05:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.882+0000 7fd66affd640 1 -- 172.21.15.49:0/1065079819 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd6640073d0 con 0x7fd66c0b3210 2024-03-14T22:05:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.883+0000 7fd66affd640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd65806a7a0 0x7fd65806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:05:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.883+0000 7fd66affd640 1 -- 172.21.15.49:0/1065079819 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd6640882f0 con 0x7fd66c0b3210 2024-03-14T22:05:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.883+0000 7fd67184a640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd65806a7a0 0x7fd65806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:05:52.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.884+0000 7fd67184a640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd65806a7a0 0x7fd65806cc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd660036000 tx=0x7fd66003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:05:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:52.887+0000 7fd66affd640 1 -- 172.21.15.49:0/1065079819 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd6640536b0 con 0x7fd66c0b3210 2024-03-14T22:05:53.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.064+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd66c070f20 con 0x7fd65806a7a0 2024-03-14T22:05:53.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.072+0000 7fd66affd640 1 -- 172.21.15.49:0/1065079819 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291081 (secure 0 0 0) 0x7fd660045410 con 0x7fd65806a7a0 2024-03-14T22:05:53.074 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:05:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd65806a7a0 msgr2=0x7fd65806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd65806a7a0 0x7fd65806cc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd660036000 tx=0x7fd66003c040 comp rx=0 tx=0).stop 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3210 msgr2=0x7fd66c1e3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3210 0x7fd66c1e3530 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c4a0 tx=0x7fd66400c960 comp rx=0 tx=0).stop 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 shutdown_connections 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fd65806a7a0 0x7fd65806cc60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd66c141be0 0x7fd66c1d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd66c0b3a90 0x7fd66c1d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 --2- 172.21.15.49:0/1065079819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd66c0b3210 0x7fd66c1e3530 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 >> 172.21.15.49:0/1065079819 conn(0x7fd66c0b9c30 msgr2=0x7fd66c13fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:05:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 shutdown_connections 2024-03-14T22:05:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:05:53.076+0000 7fd6742af640 1 -- 172.21.15.49:0/1065079819 wait complete. 2024-03-14T22:05:53.085 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":1577,"stamp":"2024-03-14T22:05:51.595187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257497981,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":4402,"num_read_kb":112073,"num_write":843876,"num_write_kb":9407945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23590,"num_omap_keys":34,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896072,"ondisk_log_size":896072,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8159732,"kb_used_data":504884,"kb_used_omap":13,"kb_used_meta":7654770,"kb_avail":746814988,"statfs":{"total":773094113280,"available":764738547712,"internally_reserved":0,"allocated":517001216,"data_stored":515697415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":7838484568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22420,"num_omap_keys":32,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003052"},"pg_stats":[{"pgid":"3.3f","version":"23'10331","reported_seq":10374,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.853007+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T22:05:25.853007+0000","last_peered":"2024-03-14T22:05:25.853007+0000","last_clean":"2024-03-14T22:05:25.853007+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T22:05:25.853007+0000","last_undegraded":"2024-03-14T22:05:25.853007+0000","last_fullsized":"2024-03-14T22:05:25.853007+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8785,"num_write_kb":106962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'9098","reported_seq":9148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.747335+0000","last_change":"2024-03-14T21:14:03.035112+0000","last_active":"2024-03-14T22:05:25.747335+0000","last_peered":"2024-03-14T22:05:25.747335+0000","last_clean":"2024-03-14T22:05:25.747335+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T22:05:25.747335+0000","last_undegraded":"2024-03-14T22:05:25.747335+0000","last_fullsized":"2024-03-14T22:05:25.747335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":7584,"num_write_kb":103945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'11564","reported_seq":11606,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.712807+0000","last_change":"2024-03-14T21:14:03.034522+0000","last_active":"2024-03-14T22:05:25.712807+0000","last_peered":"2024-03-14T22:05:25.712807+0000","last_clean":"2024-03-14T22:05:25.712807+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T22:05:25.712807+0000","last_undegraded":"2024-03-14T22:05:25.712807+0000","last_fullsized":"2024-03-14T22:05:25.712807+0000","mapping_epoch":19,"log_start":"23'1500","ondisk_log_start":"23'1500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1274,"num_write":9914,"num_write_kb":127294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'10102","reported_seq":10149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.803339+0000","last_change":"2024-03-14T21:14:03.034925+0000","last_active":"2024-03-14T22:05:25.803339+0000","last_peered":"2024-03-14T22:05:25.803339+0000","last_clean":"2024-03-14T22:05:25.803339+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T22:05:25.803339+0000","last_undegraded":"2024-03-14T22:05:25.803339+0000","last_fullsized":"2024-03-14T22:05:25.803339+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8407,"num_write_kb":110879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'9608","reported_seq":9643,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.748202+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T22:05:25.748202+0000","last_peered":"2024-03-14T22:05:25.748202+0000","last_clean":"2024-03-14T22:05:25.748202+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T22:05:25.748202+0000","last_undegraded":"2024-03-14T22:05:25.748202+0000","last_fullsized":"2024-03-14T22:05:25.748202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9608,"log_dups_size":0,"ondisk_log_size":9608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8008,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'9645","reported_seq":9686,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.885051+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T22:05:25.885051+0000","last_peered":"2024-03-14T22:05:25.885051+0000","last_clean":"2024-03-14T22:05:25.885051+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T22:05:25.885051+0000","last_undegraded":"2024-03-14T22:05:25.885051+0000","last_fullsized":"2024-03-14T22:05:25.885051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9645,"log_dups_size":0,"ondisk_log_size":9645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":8054,"num_write_kb":114907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10037","reported_seq":10074,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.757404+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T22:05:25.757404+0000","last_peered":"2024-03-14T22:05:25.757404+0000","last_clean":"2024-03-14T22:05:25.757404+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T22:05:25.757404+0000","last_undegraded":"2024-03-14T22:05:25.757404+0000","last_fullsized":"2024-03-14T22:05:25.757404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":8423,"num_write_kb":104789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'11196","reported_seq":11237,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.859079+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T22:05:25.859079+0000","last_peered":"2024-03-14T22:05:25.859079+0000","last_clean":"2024-03-14T22:05:25.859079+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T22:05:25.859079+0000","last_undegraded":"2024-03-14T22:05:25.859079+0000","last_fullsized":"2024-03-14T22:05:25.859079+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":9605,"num_write_kb":125652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'9017","reported_seq":9055,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.837419+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T22:05:25.837419+0000","last_peered":"2024-03-14T22:05:25.837419+0000","last_clean":"2024-03-14T22:05:25.837419+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T22:05:25.837419+0000","last_undegraded":"2024-03-14T22:05:25.837419+0000","last_fullsized":"2024-03-14T22:05:25.837419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9017,"log_dups_size":0,"ondisk_log_size":9017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7381,"num_write_kb":104568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'9539","reported_seq":9576,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.843247+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T22:05:25.843247+0000","last_peered":"2024-03-14T22:05:25.843247+0000","last_clean":"2024-03-14T22:05:25.843247+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T22:05:25.843247+0000","last_undegraded":"2024-03-14T22:05:25.843247+0000","last_fullsized":"2024-03-14T22:05:25.843247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7843,"num_write_kb":113016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'9229","reported_seq":9271,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.828048+0000","last_change":"2024-03-14T21:14:03.033642+0000","last_active":"2024-03-14T22:05:25.828048+0000","last_peered":"2024-03-14T22:05:25.828048+0000","last_clean":"2024-03-14T22:05:25.828048+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T22:05:25.828048+0000","last_undegraded":"2024-03-14T22:05:25.828048+0000","last_fullsized":"2024-03-14T22:05:25.828048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9229,"log_dups_size":0,"ondisk_log_size":9229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7677,"num_write_kb":104837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'10311","reported_seq":10358,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:36.777685+0000","last_change":"2024-03-14T22:05:36.777822+0000","last_active":"2024-03-14T22:05:36.777685+0000","last_peered":"2024-03-14T22:05:36.777685+0000","last_clean":"2024-03-14T22:05:36.777685+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T22:05:36.777685+0000","last_undegraded":"2024-03-14T22:05:36.777685+0000","last_fullsized":"2024-03-14T22:05:36.777685+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10311","last_scrub_stamp":"2024-03-14T22:05:36.777662+0000","last_deep_scrub":"23'10311","last_deep_scrub_stamp":"2024-03-14T22:05:36.777662+0000","last_clean_scrub_stamp":"2024-03-14T22:05:36.777662+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:58:06.567940+0000","scrub_duration":0.019761048999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8730,"num_write_kb":105734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'8864","reported_seq":8903,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.809302+0000","last_change":"2024-03-14T21:14:03.041434+0000","last_active":"2024-03-14T22:05:25.809302+0000","last_peered":"2024-03-14T22:05:25.809302+0000","last_clean":"2024-03-14T22:05:25.809302+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T22:05:25.809302+0000","last_undegraded":"2024-03-14T22:05:25.809302+0000","last_fullsized":"2024-03-14T22:05:25.809302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8864,"log_dups_size":0,"ondisk_log_size":8864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":7264,"num_write_kb":95419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'10416","reported_seq":10457,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:29.211247+0000","last_change":"2024-03-14T21:14:03.028696+0000","last_active":"2024-03-14T22:05:29.211247+0000","last_peered":"2024-03-14T22:05:29.211247+0000","last_clean":"2024-03-14T22:05:29.211247+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T22:05:29.211247+0000","last_undegraded":"2024-03-14T22:05:29.211247+0000","last_fullsized":"2024-03-14T22:05:29.211247+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8793,"num_write_kb":111405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'9704","reported_seq":9743,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.901656+0000","last_change":"2024-03-14T21:14:03.033099+0000","last_active":"2024-03-14T22:05:25.901656+0000","last_peered":"2024-03-14T22:05:25.901656+0000","last_clean":"2024-03-14T22:05:25.901656+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T22:05:25.901656+0000","last_undegraded":"2024-03-14T22:05:25.901656+0000","last_fullsized":"2024-03-14T22:05:25.901656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9704,"log_dups_size":0,"ondisk_log_size":9704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8012,"num_write_kb":95820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'9661","reported_seq":9710,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.830064+0000","last_change":"2024-03-14T21:14:03.027791+0000","last_active":"2024-03-14T22:05:25.830064+0000","last_peered":"2024-03-14T22:05:25.830064+0000","last_clean":"2024-03-14T22:05:25.830064+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T22:05:25.830064+0000","last_undegraded":"2024-03-14T22:05:25.830064+0000","last_fullsized":"2024-03-14T22:05:25.830064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9661,"log_dups_size":0,"ondisk_log_size":9661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3572,"num_write":8076,"num_write_kb":107857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8662","reported_seq":8700,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.771356+0000","last_change":"2024-03-14T21:14:44.859268+0000","last_active":"2024-03-14T22:05:25.771356+0000","last_peered":"2024-03-14T22:05:25.771356+0000","last_clean":"2024-03-14T22:05:25.771356+0000","last_became_active":"2024-03-14T21:14:44.858906+0000","last_became_peered":"2024-03-14T21:14:44.858906+0000","last_unstale":"2024-03-14T22:05:25.771356+0000","last_undegraded":"2024-03-14T22:05:25.771356+0000","last_fullsized":"2024-03-14T22:05:25.771356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8662,"log_dups_size":0,"ondisk_log_size":8662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1528,"num_write":7020,"num_write_kb":102847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10199","reported_seq":10247,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:46.076003+0000","last_change":"2024-03-14T22:05:46.076125+0000","last_active":"2024-03-14T22:05:46.076003+0000","last_peered":"2024-03-14T22:05:46.076003+0000","last_clean":"2024-03-14T22:05:46.076003+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T22:05:46.076003+0000","last_undegraded":"2024-03-14T22:05:46.076003+0000","last_fullsized":"2024-03-14T22:05:46.076003+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10199","last_scrub_stamp":"2024-03-14T22:05:46.075978+0000","last_deep_scrub":"23'10199","last_deep_scrub_stamp":"2024-03-14T22:05:46.075978+0000","last_clean_scrub_stamp":"2024-03-14T22:05:46.075978+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:28.104989+0000","scrub_duration":0.011461096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8552,"num_write_kb":105487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'10521","reported_seq":10556,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.822892+0000","last_change":"2024-03-14T21:14:03.033989+0000","last_active":"2024-03-14T22:05:25.822892+0000","last_peered":"2024-03-14T22:05:25.822892+0000","last_clean":"2024-03-14T22:05:25.822892+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T22:05:25.822892+0000","last_undegraded":"2024-03-14T22:05:25.822892+0000","last_fullsized":"2024-03-14T22:05:25.822892+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":118219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'10900","reported_seq":10940,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.956697+0000","last_change":"2024-03-14T21:14:03.035722+0000","last_active":"2024-03-14T22:05:31.956697+0000","last_peered":"2024-03-14T22:05:31.956697+0000","last_clean":"2024-03-14T22:05:31.956697+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T22:05:31.956697+0000","last_undegraded":"2024-03-14T22:05:31.956697+0000","last_fullsized":"2024-03-14T22:05:31.956697+0000","mapping_epoch":19,"log_start":"23'800","ondisk_log_start":"23'800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":9315,"num_write_kb":117297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'10962","reported_seq":11010,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:29.211289+0000","last_change":"2024-03-14T21:14:03.035067+0000","last_active":"2024-03-14T22:05:29.211289+0000","last_peered":"2024-03-14T22:05:29.211289+0000","last_clean":"2024-03-14T22:05:29.211289+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T22:05:29.211289+0000","last_undegraded":"2024-03-14T22:05:29.211289+0000","last_fullsized":"2024-03-14T22:05:29.211289+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3050,"num_write":9338,"num_write_kb":109552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10723","reported_seq":10761,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.501353+0000","last_change":"2024-03-14T21:14:03.031746+0000","last_active":"2024-03-14T22:05:31.501353+0000","last_peered":"2024-03-14T22:05:31.501353+0000","last_clean":"2024-03-14T22:05:31.501353+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T22:05:31.501353+0000","last_undegraded":"2024-03-14T22:05:31.501353+0000","last_fullsized":"2024-03-14T22:05:31.501353+0000","mapping_epoch":19,"log_start":"23'700","ondisk_log_start":"23'700","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":9066,"num_write_kb":118743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'10565","reported_seq":10603,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:30.583638+0000","last_change":"2024-03-14T21:14:03.041296+0000","last_active":"2024-03-14T22:05:30.583638+0000","last_peered":"2024-03-14T22:05:30.583638+0000","last_clean":"2024-03-14T22:05:30.583638+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T22:05:30.583638+0000","last_undegraded":"2024-03-14T22:05:30.583638+0000","last_fullsized":"2024-03-14T22:05:30.583638+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8950,"num_write_kb":109690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'11180","reported_seq":11233,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:49.737590+0000","last_change":"2024-03-14T22:05:49.737865+0000","last_active":"2024-03-14T22:05:49.737590+0000","last_peered":"2024-03-14T22:05:49.737590+0000","last_clean":"2024-03-14T22:05:49.737590+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T22:05:49.737590+0000","last_undegraded":"2024-03-14T22:05:49.737590+0000","last_fullsized":"2024-03-14T22:05:49.737590+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11180","last_scrub_stamp":"2024-03-14T22:05:49.737563+0000","last_deep_scrub":"23'11180","last_deep_scrub_stamp":"2024-03-14T22:05:49.737563+0000","last_clean_scrub_stamp":"2024-03-14T22:05:49.737563+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:00.522606+0000","scrub_duration":0.013646538999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1530,"num_write":9571,"num_write_kb":127549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'10195","reported_seq":10233,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.859876+0000","last_change":"2024-03-14T21:14:03.043177+0000","last_active":"2024-03-14T22:05:25.859876+0000","last_peered":"2024-03-14T22:05:25.859876+0000","last_clean":"2024-03-14T22:05:25.859876+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T22:05:25.859876+0000","last_undegraded":"2024-03-14T22:05:25.859876+0000","last_fullsized":"2024-03-14T22:05:25.859876+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":8582,"num_write_kb":110246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'12334","reported_seq":12376,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.835995+0000","last_change":"2024-03-14T21:14:03.039249+0000","last_active":"2024-03-14T22:05:25.835995+0000","last_peered":"2024-03-14T22:05:25.835995+0000","last_clean":"2024-03-14T22:05:25.835995+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T22:05:25.835995+0000","last_undegraded":"2024-03-14T22:05:25.835995+0000","last_fullsized":"2024-03-14T22:05:25.835995+0000","mapping_epoch":19,"log_start":"23'2300","ondisk_log_start":"23'2300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10697,"num_write_kb":121303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'9243","reported_seq":9287,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.671635+0000","last_change":"2024-03-14T21:14:03.031709+0000","last_active":"2024-03-14T22:05:25.671635+0000","last_peered":"2024-03-14T22:05:25.671635+0000","last_clean":"2024-03-14T22:05:25.671635+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T22:05:25.671635+0000","last_undegraded":"2024-03-14T22:05:25.671635+0000","last_fullsized":"2024-03-14T22:05:25.671635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7601,"num_write_kb":100407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'10312","reported_seq":10359,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:49.938468+0000","last_change":"2024-03-14T22:05:49.938608+0000","last_active":"2024-03-14T22:05:49.938468+0000","last_peered":"2024-03-14T22:05:49.938468+0000","last_clean":"2024-03-14T22:05:49.938468+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T22:05:49.938468+0000","last_undegraded":"2024-03-14T22:05:49.938468+0000","last_fullsized":"2024-03-14T22:05:49.938468+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10312","last_scrub_stamp":"2024-03-14T22:05:49.938442+0000","last_deep_scrub":"23'10312","last_deep_scrub_stamp":"2024-03-14T22:05:49.938442+0000","last_clean_scrub_stamp":"2024-03-14T22:05:49.938442+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:20.664950+0000","scrub_duration":0.012238142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":108137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'11957","reported_seq":12006,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.874501+0000","last_change":"2024-03-14T21:14:44.746590+0000","last_active":"2024-03-14T22:05:25.874501+0000","last_peered":"2024-03-14T22:05:25.874501+0000","last_clean":"2024-03-14T22:05:25.874501+0000","last_became_active":"2024-03-14T21:14:44.746271+0000","last_became_peered":"2024-03-14T21:14:44.746271+0000","last_unstale":"2024-03-14T22:05:25.874501+0000","last_undegraded":"2024-03-14T22:05:25.874501+0000","last_fullsized":"2024-03-14T22:05:25.874501+0000","mapping_epoch":22,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":10306,"num_write_kb":120667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'11954","reported_seq":11997,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.887864+0000","last_change":"2024-03-14T21:14:03.038525+0000","last_active":"2024-03-14T22:05:25.887864+0000","last_peered":"2024-03-14T22:05:25.887864+0000","last_clean":"2024-03-14T22:05:25.887864+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T22:05:25.887864+0000","last_undegraded":"2024-03-14T22:05:25.887864+0000","last_fullsized":"2024-03-14T22:05:25.887864+0000","mapping_epoch":19,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10416,"num_write_kb":126678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'11296","reported_seq":11355,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.103842+0000","last_change":"2024-03-14T22:05:47.103982+0000","last_active":"2024-03-14T22:05:47.103842+0000","last_peered":"2024-03-14T22:05:47.103842+0000","last_clean":"2024-03-14T22:05:47.103842+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T22:05:47.103842+0000","last_undegraded":"2024-03-14T22:05:47.103842+0000","last_fullsized":"2024-03-14T22:05:47.103842+0000","mapping_epoch":19,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11296","last_scrub_stamp":"2024-03-14T22:05:47.103819+0000","last_deep_scrub":"23'11296","last_deep_scrub_stamp":"2024-03-14T22:05:47.103819+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.103819+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:17.830327+0000","scrub_duration":0.013053291999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2806,"num_write":9622,"num_write_kb":118224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'8637","reported_seq":8675,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.822407+0000","last_change":"2024-03-14T21:14:03.034317+0000","last_active":"2024-03-14T22:05:25.822407+0000","last_peered":"2024-03-14T22:05:25.822407+0000","last_clean":"2024-03-14T22:05:25.822407+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T22:05:25.822407+0000","last_undegraded":"2024-03-14T22:05:25.822407+0000","last_fullsized":"2024-03-14T22:05:25.822407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":7051,"num_write_kb":105885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'9053","reported_seq":9101,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.818239+0000","last_change":"2024-03-14T22:05:37.818351+0000","last_active":"2024-03-14T22:05:37.818239+0000","last_peered":"2024-03-14T22:05:37.818239+0000","last_clean":"2024-03-14T22:05:37.818239+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T22:05:37.818239+0000","last_undegraded":"2024-03-14T22:05:37.818239+0000","last_fullsized":"2024-03-14T22:05:37.818239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9053","last_scrub_stamp":"2024-03-14T22:05:37.818216+0000","last_deep_scrub":"23'9053","last_deep_scrub_stamp":"2024-03-14T22:05:37.818216+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.818216+0000","objects_scrubbed":0,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:12:07.513721+0000","scrub_duration":0.014777571999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":7364,"num_write_kb":104642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'10865","reported_seq":10910,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.106755+0000","last_change":"2024-03-14T22:05:37.106859+0000","last_active":"2024-03-14T22:05:37.106755+0000","last_peered":"2024-03-14T22:05:37.106755+0000","last_clean":"2024-03-14T22:05:37.106755+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T22:05:37.106755+0000","last_undegraded":"2024-03-14T22:05:37.106755+0000","last_fullsized":"2024-03-14T22:05:37.106755+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10865","last_scrub_stamp":"2024-03-14T22:05:37.106731+0000","last_deep_scrub":"23'10865","last_deep_scrub_stamp":"2024-03-14T22:05:37.106731+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.106731+0000","objects_scrubbed":4,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:08.649923+0000","scrub_duration":0.054954579000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10855,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'9539","reported_seq":9590,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.102174+0000","last_change":"2024-03-14T22:05:43.102310+0000","last_active":"2024-03-14T22:05:43.102174+0000","last_peered":"2024-03-14T22:05:43.102174+0000","last_clean":"2024-03-14T22:05:43.102174+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T22:05:43.102174+0000","last_undegraded":"2024-03-14T22:05:43.102174+0000","last_fullsized":"2024-03-14T22:05:43.102174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9539","last_scrub_stamp":"2024-03-14T22:05:43.102152+0000","last_deep_scrub":"23'9539","last_deep_scrub_stamp":"2024-03-14T22:05:43.102152+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.102152+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:27.735439+0000","scrub_duration":0.010746225999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":7923,"num_write_kb":107722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10915","reported_seq":10960,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.455431+0000","last_change":"2024-03-14T22:05:43.455532+0000","last_active":"2024-03-14T22:05:43.455431+0000","last_peered":"2024-03-14T22:05:43.455431+0000","last_clean":"2024-03-14T22:05:43.455431+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T22:05:43.455431+0000","last_undegraded":"2024-03-14T22:05:43.455431+0000","last_fullsized":"2024-03-14T22:05:43.455431+0000","mapping_epoch":18,"log_start":"23'900","ondisk_log_start":"23'900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10915","last_scrub_stamp":"2024-03-14T22:05:43.455407+0000","last_deep_scrub":"23'10915","last_deep_scrub_stamp":"2024-03-14T22:05:43.455407+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.455407+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:28.088694+0000","scrub_duration":0.005473159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":10937,"num_write_kb":77181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'7037","reported_seq":7083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.042078+0000","last_change":"2024-03-14T22:05:38.042187+0000","last_active":"2024-03-14T22:05:38.042078+0000","last_peered":"2024-03-14T22:05:38.042078+0000","last_clean":"2024-03-14T22:05:38.042078+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T22:05:38.042078+0000","last_undegraded":"2024-03-14T22:05:38.042078+0000","last_fullsized":"2024-03-14T22:05:38.042078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7037","last_scrub_stamp":"2024-03-14T22:05:38.042054+0000","last_deep_scrub":"23'7037","last_deep_scrub_stamp":"2024-03-14T22:05:38.042054+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.042054+0000","objects_scrubbed":1,"log_size":7037,"log_dups_size":0,"ondisk_log_size":7037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:22.675341+0000","scrub_duration":0.019933317999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7046,"num_write_kb":58697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'11487","reported_seq":11522,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.847700+0000","last_change":"2024-03-14T21:14:03.037382+0000","last_active":"2024-03-14T22:05:25.847700+0000","last_peered":"2024-03-14T22:05:25.847700+0000","last_clean":"2024-03-14T22:05:25.847700+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T22:05:25.847700+0000","last_undegraded":"2024-03-14T22:05:25.847700+0000","last_fullsized":"2024-03-14T22:05:25.847700+0000","mapping_epoch":19,"log_start":"23'1400","ondisk_log_start":"23'1400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":9890,"num_write_kb":106254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'11077","reported_seq":11123,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:40.604311+0000","last_change":"2024-03-14T22:05:40.604410+0000","last_active":"2024-03-14T22:05:40.604311+0000","last_peered":"2024-03-14T22:05:40.604311+0000","last_clean":"2024-03-14T22:05:40.604311+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T22:05:40.604311+0000","last_undegraded":"2024-03-14T22:05:40.604311+0000","last_fullsized":"2024-03-14T22:05:40.604311+0000","mapping_epoch":18,"log_start":"23'1000","ondisk_log_start":"23'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11077","last_scrub_stamp":"2024-03-14T22:05:40.604286+0000","last_deep_scrub":"23'11077","last_deep_scrub_stamp":"2024-03-14T22:05:40.604286+0000","last_clean_scrub_stamp":"2024-03-14T22:05:40.604286+0000","objects_scrubbed":1,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:30.626988+0000","scrub_duration":0.021895357000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11078,"num_write_kb":81064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'10475","reported_seq":10520,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.817331+0000","last_change":"2024-03-14T21:14:03.035628+0000","last_active":"2024-03-14T22:05:25.817331+0000","last_peered":"2024-03-14T22:05:25.817331+0000","last_clean":"2024-03-14T22:05:25.817331+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T22:05:25.817331+0000","last_undegraded":"2024-03-14T22:05:25.817331+0000","last_fullsized":"2024-03-14T22:05:25.817331+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2800,"num_write":8909,"num_write_kb":114509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8286","reported_seq":8926,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.430025+0000","last_change":"2024-03-14T21:14:02.662876+0000","last_active":"2024-03-14T22:05:23.430025+0000","last_peered":"2024-03-14T22:05:23.430025+0000","last_clean":"2024-03-14T22:05:23.430025+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T22:05:23.430025+0000","last_undegraded":"2024-03-14T22:05:23.430025+0000","last_fullsized":"2024-03-14T22:05:23.430025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8286,"log_dups_size":0,"ondisk_log_size":8286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":607,"num_read_kb":4494,"num_write":8292,"num_write_kb":66794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10203","reported_seq":10242,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.760726+0000","last_change":"2024-03-14T21:14:03.031762+0000","last_active":"2024-03-14T22:05:25.760726+0000","last_peered":"2024-03-14T22:05:25.760726+0000","last_clean":"2024-03-14T22:05:25.760726+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T22:05:25.760726+0000","last_undegraded":"2024-03-14T22:05:25.760726+0000","last_fullsized":"2024-03-14T22:05:25.760726+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8609,"num_write_kb":104201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'10695","reported_seq":10741,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.657443+0000","last_change":"2024-03-14T22:05:39.657538+0000","last_active":"2024-03-14T22:05:39.657443+0000","last_peered":"2024-03-14T22:05:39.657443+0000","last_clean":"2024-03-14T22:05:39.657443+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T22:05:39.657443+0000","last_undegraded":"2024-03-14T22:05:39.657443+0000","last_fullsized":"2024-03-14T22:05:39.657443+0000","mapping_epoch":18,"log_start":"23'600","ondisk_log_start":"23'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10695","last_scrub_stamp":"2024-03-14T22:05:39.657419+0000","last_deep_scrub":"23'10695","last_deep_scrub_stamp":"2024-03-14T22:05:39.657419+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.657419+0000","objects_scrubbed":6,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:10.383927+0000","scrub_duration":0.080830166999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":92935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'9099","reported_seq":9137,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.773805+0000","last_change":"2024-03-14T21:14:03.030759+0000","last_active":"2024-03-14T22:05:25.773805+0000","last_peered":"2024-03-14T22:05:25.773805+0000","last_clean":"2024-03-14T22:05:25.773805+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T22:05:25.773805+0000","last_undegraded":"2024-03-14T22:05:25.773805+0000","last_fullsized":"2024-03-14T22:05:25.773805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9099,"log_dups_size":0,"ondisk_log_size":9099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7455,"num_write_kb":104328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9149","reported_seq":9184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:26.341527+0000","last_change":"2024-03-14T21:14:02.654467+0000","last_active":"2024-03-14T22:05:26.341527+0000","last_peered":"2024-03-14T22:05:26.341527+0000","last_clean":"2024-03-14T22:05:26.341527+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T22:05:26.341527+0000","last_undegraded":"2024-03-14T22:05:26.341527+0000","last_fullsized":"2024-03-14T22:05:26.341527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":9149,"log_dups_size":0,"ondisk_log_size":9149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9159,"num_write_kb":60302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11329","reported_seq":11385,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.792145+0000","last_change":"2024-03-14T22:05:34.792145+0000","last_active":"2024-03-14T22:05:34.792145+0000","last_peered":"2024-03-14T22:05:34.792145+0000","last_clean":"2024-03-14T22:05:34.792145+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T22:05:34.792145+0000","last_undegraded":"2024-03-14T22:05:34.792145+0000","last_fullsized":"2024-03-14T22:05:34.792145+0000","mapping_epoch":19,"log_start":"23'1300","ondisk_log_start":"23'1300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11329","last_scrub_stamp":"2024-03-14T22:05:34.791977+0000","last_deep_scrub":"23'11329","last_deep_scrub_stamp":"2024-03-14T22:05:34.791977+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.791977+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:36:18.331402+0000","scrub_duration":0.011767835000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2542,"num_write":9699,"num_write_kb":113170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'13354","reported_seq":13955,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.451109+0000","last_change":"2024-03-14T22:05:38.451212+0000","last_active":"2024-03-14T22:05:38.451109+0000","last_peered":"2024-03-14T22:05:38.451109+0000","last_clean":"2024-03-14T22:05:38.451109+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T22:05:38.451109+0000","last_undegraded":"2024-03-14T22:05:38.451109+0000","last_fullsized":"2024-03-14T22:05:38.451109+0000","mapping_epoch":18,"log_start":"23'3300","ondisk_log_start":"23'3300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13354","last_scrub_stamp":"2024-03-14T22:05:38.451087+0000","last_deep_scrub":"23'13354","last_deep_scrub_stamp":"2024-03-14T22:05:38.451087+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.451087+0000","objects_scrubbed":4,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:50.713752+0000","scrub_duration":0.026780381999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":557,"num_read_kb":4455,"num_write":13349,"num_write_kb":107014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'9106","reported_seq":9153,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.805465+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T22:05:25.805465+0000","last_peered":"2024-03-14T22:05:25.805465+0000","last_clean":"2024-03-14T22:05:25.805465+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T22:05:25.805465+0000","last_undegraded":"2024-03-14T22:05:25.805465+0000","last_fullsized":"2024-03-14T22:05:25.805465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9106,"log_dups_size":0,"ondisk_log_size":9106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3318,"num_write":7536,"num_write_kb":95248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'8011","reported_seq":8058,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.179662+0000","last_change":"2024-03-14T22:05:37.179778+0000","last_active":"2024-03-14T22:05:37.179662+0000","last_peered":"2024-03-14T22:05:37.179662+0000","last_clean":"2024-03-14T22:05:37.179662+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T22:05:37.179662+0000","last_undegraded":"2024-03-14T22:05:37.179662+0000","last_fullsized":"2024-03-14T22:05:37.179662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8011","last_scrub_stamp":"2024-03-14T22:05:37.179641+0000","last_deep_scrub":"23'8011","last_deep_scrub_stamp":"2024-03-14T22:05:37.179641+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.179641+0000","objects_scrubbed":2,"log_size":8011,"log_dups_size":0,"ondisk_log_size":8011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:49.442306+0000","scrub_duration":0.011524995999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8008,"num_write_kb":62700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'9649","reported_seq":9689,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.805938+0000","last_change":"2024-03-14T21:14:03.034050+0000","last_active":"2024-03-14T22:05:25.805938+0000","last_peered":"2024-03-14T22:05:25.805938+0000","last_clean":"2024-03-14T22:05:25.805938+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T22:05:25.805938+0000","last_undegraded":"2024-03-14T22:05:25.805938+0000","last_fullsized":"2024-03-14T22:05:25.805938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9649,"log_dups_size":0,"ondisk_log_size":9649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8033,"num_write_kb":100651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5758","reported_seq":5805,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:45.712184+0000","last_change":"2024-03-14T22:05:45.712286+0000","last_active":"2024-03-14T22:05:45.712184+0000","last_peered":"2024-03-14T22:05:45.712184+0000","last_clean":"2024-03-14T22:05:45.712184+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T22:05:45.712184+0000","last_undegraded":"2024-03-14T22:05:45.712184+0000","last_fullsized":"2024-03-14T22:05:45.712184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5758","last_scrub_stamp":"2024-03-14T22:05:45.712161+0000","last_deep_scrub":"23'5758","last_deep_scrub_stamp":"2024-03-14T22:05:45.712161+0000","last_clean_scrub_stamp":"2024-03-14T22:05:45.712161+0000","objects_scrubbed":2,"log_size":5758,"log_dups_size":0,"ondisk_log_size":5758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:45.011446+0000","scrub_duration":0.027383683999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5754,"num_write_kb":48600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'9764","reported_seq":9801,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.764713+0000","last_change":"2024-03-14T21:14:03.032694+0000","last_active":"2024-03-14T22:05:25.764713+0000","last_peered":"2024-03-14T22:05:25.764713+0000","last_clean":"2024-03-14T22:05:25.764713+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T22:05:25.764713+0000","last_undegraded":"2024-03-14T22:05:25.764713+0000","last_fullsized":"2024-03-14T22:05:25.764713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9764,"log_dups_size":0,"ondisk_log_size":9764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8135,"num_write_kb":106503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5852","reported_seq":5899,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.442569+0000","last_change":"2024-03-14T22:05:42.442683+0000","last_active":"2024-03-14T22:05:42.442569+0000","last_peered":"2024-03-14T22:05:42.442569+0000","last_clean":"2024-03-14T22:05:42.442569+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T22:05:42.442569+0000","last_undegraded":"2024-03-14T22:05:42.442569+0000","last_fullsized":"2024-03-14T22:05:42.442569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5852","last_scrub_stamp":"2024-03-14T22:05:42.442545+0000","last_deep_scrub":"23'5852","last_deep_scrub_stamp":"2024-03-14T22:05:42.442545+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.442545+0000","objects_scrubbed":1,"log_size":5852,"log_dups_size":0,"ondisk_log_size":5852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:13.985738+0000","scrub_duration":0.016901003000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":52920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'9086","reported_seq":9131,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.822222+0000","last_change":"2024-03-14T21:14:03.034246+0000","last_active":"2024-03-14T22:05:25.822222+0000","last_peered":"2024-03-14T22:05:25.822222+0000","last_clean":"2024-03-14T22:05:25.822222+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T22:05:25.822222+0000","last_undegraded":"2024-03-14T22:05:25.822222+0000","last_fullsized":"2024-03-14T22:05:25.822222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9086,"log_dups_size":0,"ondisk_log_size":9086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2548,"num_write":7495,"num_write_kb":101746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8334","reported_seq":8381,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:30.179949+0000","last_change":"2024-03-14T22:05:30.180079+0000","last_active":"2024-03-14T22:05:30.179949+0000","last_peered":"2024-03-14T22:05:30.179949+0000","last_clean":"2024-03-14T22:05:30.179949+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T22:05:30.179949+0000","last_undegraded":"2024-03-14T22:05:30.179949+0000","last_fullsized":"2024-03-14T22:05:30.179949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8334","last_scrub_stamp":"2024-03-14T22:05:30.179928+0000","last_deep_scrub":"23'8334","last_deep_scrub_stamp":"2024-03-14T22:05:30.179928+0000","last_clean_scrub_stamp":"2024-03-14T22:05:30.179928+0000","objects_scrubbed":8,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:42.442593+0000","scrub_duration":0.057801277999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16724139,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":8340,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'10136","reported_seq":10172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.901810+0000","last_change":"2024-03-14T21:14:44.859568+0000","last_active":"2024-03-14T22:05:25.901810+0000","last_peered":"2024-03-14T22:05:25.901810+0000","last_clean":"2024-03-14T22:05:25.901810+0000","last_became_active":"2024-03-14T21:14:44.859185+0000","last_became_peered":"2024-03-14T21:14:44.859185+0000","last_unstale":"2024-03-14T22:05:25.901810+0000","last_undegraded":"2024-03-14T22:05:25.901810+0000","last_fullsized":"2024-03-14T22:05:25.901810+0000","mapping_epoch":22,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":8491,"num_write_kb":113136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"23'8049","reported_seq":8096,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.782674+0000","last_change":"2024-03-14T22:05:31.782835+0000","last_active":"2024-03-14T22:05:31.782674+0000","last_peered":"2024-03-14T22:05:31.782674+0000","last_clean":"2024-03-14T22:05:31.782674+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T22:05:31.782674+0000","last_undegraded":"2024-03-14T22:05:31.782674+0000","last_fullsized":"2024-03-14T22:05:31.782674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8049","last_scrub_stamp":"2024-03-14T22:05:31.782651+0000","last_deep_scrub":"23'8049","last_deep_scrub_stamp":"2024-03-14T22:05:31.782651+0000","last_clean_scrub_stamp":"2024-03-14T22:05:31.782651+0000","objects_scrubbed":3,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:57:34.557091+0000","scrub_duration":0.036800939999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8051,"num_write_kb":62569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'9555","reported_seq":9610,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.307278+0000","last_change":"2024-03-14T22:05:34.307278+0000","last_active":"2024-03-14T22:05:34.307278+0000","last_peered":"2024-03-14T22:05:34.307278+0000","last_clean":"2024-03-14T22:05:34.307278+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T22:05:34.307278+0000","last_undegraded":"2024-03-14T22:05:34.307278+0000","last_fullsized":"2024-03-14T22:05:34.307278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9555","last_scrub_stamp":"2024-03-14T22:05:34.307098+0000","last_deep_scrub":"23'9555","last_deep_scrub_stamp":"2024-03-14T22:05:34.307098+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.307098+0000","objects_scrubbed":0,"log_size":9555,"log_dups_size":0,"ondisk_log_size":9555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:18.940385+0000","scrub_duration":0.012363498000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2042,"num_write":7926,"num_write_kb":109666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'10891","reported_seq":10939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.205897+0000","last_change":"2024-03-14T22:05:39.205897+0000","last_active":"2024-03-14T22:05:39.205897+0000","last_peered":"2024-03-14T22:05:39.205897+0000","last_clean":"2024-03-14T22:05:39.205897+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T22:05:39.205897+0000","last_undegraded":"2024-03-14T22:05:39.205897+0000","last_fullsized":"2024-03-14T22:05:39.205897+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10891","last_scrub_stamp":"2024-03-14T22:05:39.205752+0000","last_deep_scrub":"23'10891","last_deep_scrub_stamp":"2024-03-14T22:05:39.205752+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.205752+0000","objects_scrubbed":2,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:13.412709+0000","scrub_duration":0.023607316999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10896,"num_write_kb":88926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9015","reported_seq":9055,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.867159+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T22:05:25.867159+0000","last_peered":"2024-03-14T22:05:25.867159+0000","last_clean":"2024-03-14T22:05:25.867159+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T22:05:25.867159+0000","last_undegraded":"2024-03-14T22:05:25.867159+0000","last_fullsized":"2024-03-14T22:05:25.867159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9015,"log_dups_size":0,"ondisk_log_size":9015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":7443,"num_write_kb":93234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'11502","reported_seq":11550,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.317932+0000","last_change":"2024-03-14T22:05:39.317932+0000","last_active":"2024-03-14T22:05:39.317932+0000","last_peered":"2024-03-14T22:05:39.317932+0000","last_clean":"2024-03-14T22:05:39.317932+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T22:05:39.317932+0000","last_undegraded":"2024-03-14T22:05:39.317932+0000","last_fullsized":"2024-03-14T22:05:39.317932+0000","mapping_epoch":18,"log_start":"23'1500","ondisk_log_start":"23'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11502","last_scrub_stamp":"2024-03-14T22:05:39.317814+0000","last_deep_scrub":"23'11502","last_deep_scrub_stamp":"2024-03-14T22:05:39.317814+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.317814+0000","objects_scrubbed":4,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:23.951100+0000","scrub_duration":0.049637907000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11487,"num_write_kb":85212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'9516","reported_seq":9558,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.501238+0000","last_change":"2024-03-14T21:14:03.034967+0000","last_active":"2024-03-14T22:05:31.501238+0000","last_peered":"2024-03-14T22:05:31.501238+0000","last_clean":"2024-03-14T22:05:31.501238+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T22:05:31.501238+0000","last_undegraded":"2024-03-14T22:05:31.501238+0000","last_fullsized":"2024-03-14T22:05:31.501238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9516,"log_dups_size":0,"ondisk_log_size":9516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":7940,"num_write_kb":115635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8906","reported_seq":8954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.143252+0000","last_change":"2024-03-14T22:05:34.143252+0000","last_active":"2024-03-14T22:05:34.143252+0000","last_peered":"2024-03-14T22:05:34.143252+0000","last_clean":"2024-03-14T22:05:34.143252+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T22:05:34.143252+0000","last_undegraded":"2024-03-14T22:05:34.143252+0000","last_fullsized":"2024-03-14T22:05:34.143252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8906","last_scrub_stamp":"2024-03-14T22:05:34.143115+0000","last_deep_scrub":"23'8906","last_deep_scrub_stamp":"2024-03-14T22:05:34.143115+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.143115+0000","objects_scrubbed":2,"log_size":8906,"log_dups_size":0,"ondisk_log_size":8906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:05.686307+0000","scrub_duration":0.021707378999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8892,"num_write_kb":67382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1170,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'9427","reported_seq":9467,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:32.411600+0000","last_change":"2024-03-14T21:14:03.031405+0000","last_active":"2024-03-14T22:05:32.411600+0000","last_peered":"2024-03-14T22:05:32.411600+0000","last_clean":"2024-03-14T22:05:32.411600+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T22:05:32.411600+0000","last_undegraded":"2024-03-14T22:05:32.411600+0000","last_fullsized":"2024-03-14T22:05:32.411600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9427,"log_dups_size":0,"ondisk_log_size":9427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7820,"num_write_kb":110543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.280614+0000","last_change":"2024-03-14T22:05:38.280716+0000","last_active":"2024-03-14T22:05:38.280614+0000","last_peered":"2024-03-14T22:05:38.280614+0000","last_clean":"2024-03-14T22:05:38.280614+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T22:05:38.280614+0000","last_undegraded":"2024-03-14T22:05:38.280614+0000","last_fullsized":"2024-03-14T22:05:38.280614+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-03-14T22:05:38.280592+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-03-14T22:05:38.280592+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.280592+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:12.487548+0000","scrub_duration":0.0062042419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'8579","reported_seq":8628,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:32.525027+0000","last_change":"2024-03-14T22:05:32.525126+0000","last_active":"2024-03-14T22:05:32.525027+0000","last_peered":"2024-03-14T22:05:32.525027+0000","last_clean":"2024-03-14T22:05:32.525027+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T22:05:32.525027+0000","last_undegraded":"2024-03-14T22:05:32.525027+0000","last_fullsized":"2024-03-14T22:05:32.525027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8579","last_scrub_stamp":"2024-03-14T22:05:32.525005+0000","last_deep_scrub":"23'8579","last_deep_scrub_stamp":"2024-03-14T22:05:32.525005+0000","last_clean_scrub_stamp":"2024-03-14T22:05:32.525005+0000","objects_scrubbed":2,"log_size":8579,"log_dups_size":0,"ondisk_log_size":8579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:06.731961+0000","scrub_duration":0.031238887999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":67724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'11133","reported_seq":11173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.819412+0000","last_change":"2024-03-14T21:14:03.038850+0000","last_active":"2024-03-14T22:05:25.819412+0000","last_peered":"2024-03-14T22:05:25.819412+0000","last_clean":"2024-03-14T22:05:25.819412+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T22:05:25.819412+0000","last_undegraded":"2024-03-14T22:05:25.819412+0000","last_fullsized":"2024-03-14T22:05:25.819412+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":9507,"num_write_kb":117178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'8676","reported_seq":8715,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.431213+0000","last_change":"2024-03-14T21:14:02.661258+0000","last_active":"2024-03-14T22:05:23.431213+0000","last_peered":"2024-03-14T22:05:23.431213+0000","last_clean":"2024-03-14T22:05:23.431213+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T22:05:23.431213+0000","last_undegraded":"2024-03-14T22:05:23.431213+0000","last_fullsized":"2024-03-14T22:05:23.431213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":8676,"log_dups_size":0,"ondisk_log_size":8676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8672,"num_write_kb":67194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10246","reported_seq":10296,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.131308+0000","last_change":"2024-03-14T22:05:42.131437+0000","last_active":"2024-03-14T22:05:42.131308+0000","last_peered":"2024-03-14T22:05:42.131308+0000","last_clean":"2024-03-14T22:05:42.131308+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T22:05:42.131308+0000","last_undegraded":"2024-03-14T22:05:42.131308+0000","last_fullsized":"2024-03-14T22:05:42.131308+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10246","last_scrub_stamp":"2024-03-14T22:05:42.131282+0000","last_deep_scrub":"23'10246","last_deep_scrub_stamp":"2024-03-14T22:05:42.131282+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.131282+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:13.674474+0000","scrub_duration":0.011884756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8656,"num_write_kb":115905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11306","reported_seq":12022,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:35.173673+0000","last_change":"2024-03-14T22:05:35.173769+0000","last_active":"2024-03-14T22:05:35.173673+0000","last_peered":"2024-03-14T22:05:35.173673+0000","last_clean":"2024-03-14T22:05:35.173673+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T22:05:35.173673+0000","last_undegraded":"2024-03-14T22:05:35.173673+0000","last_fullsized":"2024-03-14T22:05:35.173673+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11306","last_scrub_stamp":"2024-03-14T22:05:35.173652+0000","last_deep_scrub":"23'11306","last_deep_scrub_stamp":"2024-03-14T22:05:35.173652+0000","last_clean_scrub_stamp":"2024-03-14T22:05:35.173652+0000","objects_scrubbed":3,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:19.806939+0000","scrub_duration":0.046066214000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":669,"num_read_kb":4516,"num_write":11296,"num_write_kb":89038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'9443","reported_seq":9484,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.783718+0000","last_change":"2024-03-14T21:14:03.034973+0000","last_active":"2024-03-14T22:05:25.783718+0000","last_peered":"2024-03-14T22:05:25.783718+0000","last_clean":"2024-03-14T22:05:25.783718+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T22:05:25.783718+0000","last_undegraded":"2024-03-14T22:05:25.783718+0000","last_fullsized":"2024-03-14T22:05:25.783718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9443,"log_dups_size":0,"ondisk_log_size":9443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7837,"num_write_kb":123513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'9296","reported_seq":9345,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.595557+0000","last_change":"2024-03-14T22:05:38.595654+0000","last_active":"2024-03-14T22:05:38.595557+0000","last_peered":"2024-03-14T22:05:38.595557+0000","last_clean":"2024-03-14T22:05:38.595557+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T22:05:38.595557+0000","last_undegraded":"2024-03-14T22:05:38.595557+0000","last_fullsized":"2024-03-14T22:05:38.595557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9296","last_scrub_stamp":"2024-03-14T22:05:38.595537+0000","last_deep_scrub":"23'9296","last_deep_scrub_stamp":"2024-03-14T22:05:38.595537+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.595537+0000","objects_scrubbed":0,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:20.624547+0000","scrub_duration":0.0060746669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9294,"num_write_kb":71830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'9546","reported_seq":9595,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.855593+0000","last_change":"2024-03-14T22:05:43.855707+0000","last_active":"2024-03-14T22:05:43.855593+0000","last_peered":"2024-03-14T22:05:43.855593+0000","last_clean":"2024-03-14T22:05:43.855593+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T22:05:43.855593+0000","last_undegraded":"2024-03-14T22:05:43.855593+0000","last_fullsized":"2024-03-14T22:05:43.855593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9546","last_scrub_stamp":"2024-03-14T22:05:43.855569+0000","last_deep_scrub":"23'9546","last_deep_scrub_stamp":"2024-03-14T22:05:43.855569+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.855569+0000","objects_scrubbed":0,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:35:00.547431+0000","scrub_duration":0.01173973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":7936,"num_write_kb":109238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'11136","reported_seq":11185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:50.431075+0000","last_change":"2024-03-14T22:05:50.431205+0000","last_active":"2024-03-14T22:05:50.431075+0000","last_peered":"2024-03-14T22:05:50.431075+0000","last_clean":"2024-03-14T22:05:50.431075+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T22:05:50.431075+0000","last_undegraded":"2024-03-14T22:05:50.431075+0000","last_fullsized":"2024-03-14T22:05:50.431075+0000","mapping_epoch":18,"log_start":"23'1100","ondisk_log_start":"23'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11136","last_scrub_stamp":"2024-03-14T22:05:50.431052+0000","last_deep_scrub":"23'11136","last_deep_scrub_stamp":"2024-03-14T22:05:50.431052+0000","last_clean_scrub_stamp":"2024-03-14T22:05:50.431052+0000","objects_scrubbed":2,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:32.460062+0000","scrub_duration":0.031083458000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11130,"num_write_kb":85841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'12171","reported_seq":12208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.885658+0000","last_change":"2024-03-14T21:14:03.031457+0000","last_active":"2024-03-14T22:05:25.885658+0000","last_peered":"2024-03-14T22:05:25.885658+0000","last_clean":"2024-03-14T22:05:25.885658+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T22:05:25.885658+0000","last_undegraded":"2024-03-14T22:05:25.885658+0000","last_fullsized":"2024-03-14T22:05:25.885658+0000","mapping_epoch":19,"log_start":"23'2100","ondisk_log_start":"23'2100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10560,"num_write_kb":124816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'9284","reported_seq":9333,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.431802+0000","last_change":"2024-03-14T22:05:47.431939+0000","last_active":"2024-03-14T22:05:47.431802+0000","last_peered":"2024-03-14T22:05:47.431802+0000","last_clean":"2024-03-14T22:05:47.431802+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T22:05:47.431802+0000","last_undegraded":"2024-03-14T22:05:47.431802+0000","last_fullsized":"2024-03-14T22:05:47.431802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9284","last_scrub_stamp":"2024-03-14T22:05:47.431780+0000","last_deep_scrub":"23'9284","last_deep_scrub_stamp":"2024-03-14T22:05:47.431780+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.431780+0000","objects_scrubbed":1,"log_size":9284,"log_dups_size":0,"ondisk_log_size":9284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:37.454481+0000","scrub_duration":0.019347982999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9294,"num_write_kb":71728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9772","reported_seq":9814,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.730809+0000","last_change":"2024-03-14T21:14:03.022606+0000","last_active":"2024-03-14T22:05:25.730809+0000","last_peered":"2024-03-14T22:05:25.730809+0000","last_clean":"2024-03-14T22:05:25.730809+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T22:05:25.730809+0000","last_undegraded":"2024-03-14T22:05:25.730809+0000","last_fullsized":"2024-03-14T22:05:25.730809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9772,"log_dups_size":0,"ondisk_log_size":9772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8091,"num_write_kb":107879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'8360","reported_seq":9001,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.381430+0000","last_change":"2024-03-14T22:05:38.381548+0000","last_active":"2024-03-14T22:05:38.381430+0000","last_peered":"2024-03-14T22:05:38.381430+0000","last_clean":"2024-03-14T22:05:38.381430+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T22:05:38.381430+0000","last_undegraded":"2024-03-14T22:05:38.381430+0000","last_fullsized":"2024-03-14T22:05:38.381430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8360","last_scrub_stamp":"2024-03-14T22:05:38.381407+0000","last_deep_scrub":"23'8360","last_deep_scrub_stamp":"2024-03-14T22:05:38.381407+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.381407+0000","objects_scrubbed":2,"log_size":8360,"log_dups_size":0,"ondisk_log_size":8360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:09.924599+0000","scrub_duration":0.064290047000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":593,"num_read_kb":4491,"num_write":9534,"num_write_kb":73072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'10081","reported_seq":10137,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.020230+0000","last_change":"2024-03-14T22:05:42.020373+0000","last_active":"2024-03-14T22:05:42.020230+0000","last_peered":"2024-03-14T22:05:42.020230+0000","last_clean":"2024-03-14T22:05:42.020230+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T22:05:42.020230+0000","last_undegraded":"2024-03-14T22:05:42.020230+0000","last_fullsized":"2024-03-14T22:05:42.020230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10081","last_scrub_stamp":"2024-03-14T22:05:42.020207+0000","last_deep_scrub":"23'10081","last_deep_scrub_stamp":"2024-03-14T22:05:42.020207+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.020207+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:16.227164+0000","scrub_duration":0.010075914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8532,"num_write_kb":114847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5882","reported_seq":5930,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.478403+0000","last_change":"2024-03-14T22:05:47.478496+0000","last_active":"2024-03-14T22:05:47.478403+0000","last_peered":"2024-03-14T22:05:47.478403+0000","last_clean":"2024-03-14T22:05:47.478403+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T22:05:47.478403+0000","last_undegraded":"2024-03-14T22:05:47.478403+0000","last_fullsized":"2024-03-14T22:05:47.478403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5882","last_scrub_stamp":"2024-03-14T22:05:47.478380+0000","last_deep_scrub":"23'5882","last_deep_scrub_stamp":"2024-03-14T22:05:47.478380+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.478380+0000","objects_scrubbed":2,"log_size":5882,"log_dups_size":0,"ondisk_log_size":5882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:21.685337+0000","scrub_duration":0.023401211000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5885,"num_write_kb":44672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17052,"num_omap_keys":21,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'10915","reported_seq":10969,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.120683+0000","last_change":"2024-03-14T22:05:47.120809+0000","last_active":"2024-03-14T22:05:47.120683+0000","last_peered":"2024-03-14T22:05:47.120683+0000","last_clean":"2024-03-14T22:05:47.120683+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T22:05:47.120683+0000","last_undegraded":"2024-03-14T22:05:47.120683+0000","last_fullsized":"2024-03-14T22:05:47.120683+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10915","last_scrub_stamp":"2024-03-14T22:05:47.120658+0000","last_deep_scrub":"23'10915","last_deep_scrub_stamp":"2024-03-14T22:05:47.120658+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.120658+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:21.327615+0000","scrub_duration":0.011394476000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":9274,"num_write_kb":112928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'4360","reported_seq":4397,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.380947+0000","last_change":"2024-03-14T21:14:02.661672+0000","last_active":"2024-03-14T22:05:23.380947+0000","last_peered":"2024-03-14T22:05:23.380947+0000","last_clean":"2024-03-14T22:05:23.380947+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T22:05:23.380947+0000","last_undegraded":"2024-03-14T22:05:23.380947+0000","last_fullsized":"2024-03-14T22:05:23.380947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4362,"num_write_kb":43693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'9887","reported_seq":9947,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:40.803702+0000","last_change":"2024-03-14T22:05:40.803829+0000","last_active":"2024-03-14T22:05:40.803702+0000","last_peered":"2024-03-14T22:05:40.803702+0000","last_clean":"2024-03-14T22:05:40.803702+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T22:05:40.803702+0000","last_undegraded":"2024-03-14T22:05:40.803702+0000","last_fullsized":"2024-03-14T22:05:40.803702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9887","last_scrub_stamp":"2024-03-14T22:05:40.803679+0000","last_deep_scrub":"23'9887","last_deep_scrub_stamp":"2024-03-14T22:05:40.803679+0000","last_clean_scrub_stamp":"2024-03-14T22:05:40.803679+0000","objects_scrubbed":0,"log_size":9887,"log_dups_size":0,"ondisk_log_size":9887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:46:25.442439+0000","scrub_duration":0.011556728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":8266,"num_write_kb":116331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'6408","reported_seq":6456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.393527+0000","last_change":"2024-03-14T22:05:43.393658+0000","last_active":"2024-03-14T22:05:43.393527+0000","last_peered":"2024-03-14T22:05:43.393527+0000","last_clean":"2024-03-14T22:05:43.393527+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T22:05:43.393527+0000","last_undegraded":"2024-03-14T22:05:43.393527+0000","last_fullsized":"2024-03-14T22:05:43.393527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6408","last_scrub_stamp":"2024-03-14T22:05:43.393505+0000","last_deep_scrub":"23'6408","last_deep_scrub_stamp":"2024-03-14T22:05:43.393505+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.393505+0000","objects_scrubbed":2,"log_size":6408,"log_dups_size":0,"ondisk_log_size":6408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:17.600461+0000","scrub_duration":0.029677097,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6417,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'10927","reported_seq":10974,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:48.972041+0000","last_change":"2024-03-14T22:05:48.972164+0000","last_active":"2024-03-14T22:05:48.972041+0000","last_peered":"2024-03-14T22:05:48.972041+0000","last_clean":"2024-03-14T22:05:48.972041+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T22:05:48.972041+0000","last_undegraded":"2024-03-14T22:05:48.972041+0000","last_fullsized":"2024-03-14T22:05:48.972041+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10927","last_scrub_stamp":"2024-03-14T22:05:48.972019+0000","last_deep_scrub":"23'10927","last_deep_scrub_stamp":"2024-03-14T22:05:48.972019+0000","last_clean_scrub_stamp":"2024-03-14T22:05:48.972019+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:31.001029+0000","scrub_duration":0.013693627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":120976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'9353","reported_seq":9401,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.295090+0000","last_change":"2024-03-14T22:05:34.295195+0000","last_active":"2024-03-14T22:05:34.295090+0000","last_peered":"2024-03-14T22:05:34.295090+0000","last_clean":"2024-03-14T22:05:34.295090+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T22:05:34.295090+0000","last_undegraded":"2024-03-14T22:05:34.295090+0000","last_fullsized":"2024-03-14T22:05:34.295090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9353","last_scrub_stamp":"2024-03-14T22:05:34.295063+0000","last_deep_scrub":"23'9353","last_deep_scrub_stamp":"2024-03-14T22:05:34.295063+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.295063+0000","objects_scrubbed":1,"log_size":9353,"log_dups_size":0,"ondisk_log_size":9353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:46.557728+0000","scrub_duration":0.019716635999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9374,"num_write_kb":71713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'10198","reported_seq":10238,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:30.583507+0000","last_change":"2024-03-14T21:14:03.036688+0000","last_active":"2024-03-14T22:05:30.583507+0000","last_peered":"2024-03-14T22:05:30.583507+0000","last_clean":"2024-03-14T22:05:30.583507+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T22:05:30.583507+0000","last_undegraded":"2024-03-14T22:05:30.583507+0000","last_fullsized":"2024-03-14T22:05:30.583507+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8519,"num_write_kb":106953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'11405","reported_seq":11854,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:46.439556+0000","last_change":"2024-03-14T22:05:46.439556+0000","last_active":"2024-03-14T22:05:46.439556+0000","last_peered":"2024-03-14T22:05:46.439556+0000","last_clean":"2024-03-14T22:05:46.439556+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T22:05:46.439556+0000","last_undegraded":"2024-03-14T22:05:46.439556+0000","last_fullsized":"2024-03-14T22:05:46.439556+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11405","last_scrub_stamp":"2024-03-14T22:05:46.439398+0000","last_deep_scrub":"23'11405","last_deep_scrub_stamp":"2024-03-14T22:05:46.439398+0000","last_clean_scrub_stamp":"2024-03-14T22:05:46.439398+0000","objects_scrubbed":5,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:17.165906+0000","scrub_duration":0.054388652000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13689588,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":402,"num_read_kb":1363,"num_write":11407,"num_write_kb":86544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":368,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'13463","reported_seq":13505,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.800968+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T22:05:25.800968+0000","last_peered":"2024-03-14T22:05:25.800968+0000","last_clean":"2024-03-14T22:05:25.800968+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T22:05:25.800968+0000","last_undegraded":"2024-03-14T22:05:25.800968+0000","last_fullsized":"2024-03-14T22:05:25.800968+0000","mapping_epoch":19,"log_start":"23'3400","ondisk_log_start":"23'3400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":11847,"num_write_kb":149794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'10677","reported_seq":10711,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.885659+0000","last_change":"2024-03-14T21:14:03.034887+0000","last_active":"2024-03-14T22:05:25.885659+0000","last_peered":"2024-03-14T22:05:25.885659+0000","last_clean":"2024-03-14T22:05:25.885659+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T22:05:25.885659+0000","last_undegraded":"2024-03-14T22:05:25.885659+0000","last_fullsized":"2024-03-14T22:05:25.885659+0000","mapping_epoch":19,"log_start":"23'600","ondisk_log_start":"23'600","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":8986,"num_write_kb":111494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6869","reported_seq":6914,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:45.414748+0000","last_change":"2024-03-14T22:05:45.414869+0000","last_active":"2024-03-14T22:05:45.414748+0000","last_peered":"2024-03-14T22:05:45.414748+0000","last_clean":"2024-03-14T22:05:45.414748+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T22:05:45.414748+0000","last_undegraded":"2024-03-14T22:05:45.414748+0000","last_fullsized":"2024-03-14T22:05:45.414748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6869","last_scrub_stamp":"2024-03-14T22:05:45.414726+0000","last_deep_scrub":"23'6869","last_deep_scrub_stamp":"2024-03-14T22:05:45.414726+0000","last_clean_scrub_stamp":"2024-03-14T22:05:45.414726+0000","objects_scrubbed":5,"log_size":6869,"log_dups_size":0,"ondisk_log_size":6869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:27.443736+0000","scrub_duration":0.057950375999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6885,"num_write_kb":66562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'11249","reported_seq":11298,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:48.149731+0000","last_change":"2024-03-14T22:05:48.149866+0000","last_active":"2024-03-14T22:05:48.149731+0000","last_peered":"2024-03-14T22:05:48.149731+0000","last_clean":"2024-03-14T22:05:48.149731+0000","last_became_active":"2024-03-14T21:14:44.859128+0000","last_became_peered":"2024-03-14T21:14:44.859128+0000","last_unstale":"2024-03-14T22:05:48.149731+0000","last_undegraded":"2024-03-14T22:05:48.149731+0000","last_fullsized":"2024-03-14T22:05:48.149731+0000","mapping_epoch":22,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11249","last_scrub_stamp":"2024-03-14T22:05:48.149707+0000","last_deep_scrub":"23'11249","last_deep_scrub_stamp":"2024-03-14T22:05:48.149707+0000","last_clean_scrub_stamp":"2024-03-14T22:05:48.149707+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:38.172409+0000","scrub_duration":0.015019927000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1780,"num_write":9572,"num_write_kb":116659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15422","reported_seq":16525,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:23.433100+0000","last_change":"2024-03-14T21:14:02.662717+0000","last_active":"2024-03-14T22:05:23.433100+0000","last_peered":"2024-03-14T22:05:23.433100+0000","last_clean":"2024-03-14T22:05:23.433100+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T22:05:23.433100+0000","last_undegraded":"2024-03-14T22:05:23.433100+0000","last_fullsized":"2024-03-14T22:05:23.433100+0000","mapping_epoch":18,"log_start":"23'5400","ondisk_log_start":"23'5400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:01.634028+0000","last_clean_scrub_stamp":"2024-03-14T21:14:01.634028+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1069,"num_read_kb":4840,"num_write":15404,"num_write_kb":112501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'10432","reported_seq":10480,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.818270+0000","last_change":"2024-03-14T21:14:03.036767+0000","last_active":"2024-03-14T22:05:25.818270+0000","last_peered":"2024-03-14T22:05:25.818270+0000","last_clean":"2024-03-14T22:05:25.818270+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T22:05:25.818270+0000","last_undegraded":"2024-03-14T22:05:25.818270+0000","last_fullsized":"2024-03-14T22:05:25.818270+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2796,"num_write":8857,"num_write_kb":109660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'8707","reported_seq":8757,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:33.083322+0000","last_change":"2024-03-14T22:05:33.083418+0000","last_active":"2024-03-14T22:05:33.083322+0000","last_peered":"2024-03-14T22:05:33.083322+0000","last_clean":"2024-03-14T22:05:33.083322+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T22:05:33.083322+0000","last_undegraded":"2024-03-14T22:05:33.083322+0000","last_fullsized":"2024-03-14T22:05:33.083322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8707","last_scrub_stamp":"2024-03-14T22:05:33.083300+0000","last_deep_scrub":"23'8707","last_deep_scrub_stamp":"2024-03-14T22:05:33.083300+0000","last_clean_scrub_stamp":"2024-03-14T22:05:33.083300+0000","objects_scrubbed":1,"log_size":8707,"log_dups_size":0,"ondisk_log_size":8707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:45.345964+0000","scrub_duration":0.015359457999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8698,"num_write_kb":67327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'11837","reported_seq":11876,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.812922+0000","last_change":"2024-03-14T21:14:03.032434+0000","last_active":"2024-03-14T22:05:25.812922+0000","last_peered":"2024-03-14T22:05:25.812922+0000","last_clean":"2024-03-14T22:05:25.812922+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T22:05:25.812922+0000","last_undegraded":"2024-03-14T22:05:25.812922+0000","last_fullsized":"2024-03-14T22:05:25.812922+0000","mapping_epoch":19,"log_start":"23'1800","ondisk_log_start":"23'1800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":10179,"num_write_kb":107836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6051","reported_seq":6100,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.157052+0000","last_change":"2024-03-14T22:05:31.157143+0000","last_active":"2024-03-14T22:05:31.157052+0000","last_peered":"2024-03-14T22:05:31.157052+0000","last_clean":"2024-03-14T22:05:31.157052+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T22:05:31.157052+0000","last_undegraded":"2024-03-14T22:05:31.157052+0000","last_fullsized":"2024-03-14T22:05:31.157052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6051","last_scrub_stamp":"2024-03-14T22:05:31.157033+0000","last_deep_scrub":"23'6051","last_deep_scrub_stamp":"2024-03-14T22:05:31.157033+0000","last_clean_scrub_stamp":"2024-03-14T22:05:31.157033+0000","objects_scrubbed":5,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:11:59.042596+0000","scrub_duration":0.052529798000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6048,"num_write_kb":48854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":345,"num_read_kb":87814,"num_write":552757,"num_write_kb":7123402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624278,"ondisk_log_size":624278,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":256907613,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":3941,"num_read_kb":24159,"num_write":290960,"num_write_kb":2282127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23590,"num_omap_keys":34,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":513884160,"data_stored":513815226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271675,"ondisk_log_size":271675,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575476,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252664,"kb_used_data":112496,"kb_used_omap":1,"kb_used_meta":1140158,"kb_avail":93119176,"statfs":{"total":96636764160,"available":95354036224,"internally_reserved":0,"allocated":115195904,"data_stored":115031593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1167522059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44600000000000001,"15min":0.48899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.19500000000000001},"max":{"1min":0.879,"5min":2.782,"15min":4.7999999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.63100000000000001,"15min":0.53200000000000003},"min":{"1min":0.29199999999999998,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":4.8860000000000001,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":0.95699999999999996}]},{"osd":1,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.51300000000000001,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":4.8410000000000002,"5min":4.8410000000000002,"15min":4.8410000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.28599999999999998,"15min":0.26400000000000001},"max":{"1min":2.2240000000000002,"5min":6.1440000000000001,"15min":6.1440000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52500000000000002,"15min":0.53200000000000003},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.25900000000000001},"max":{"1min":0.80300000000000005,"5min":3.8210000000000002,"15min":4.8310000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.63800000000000001,"15min":0.55300000000000005},"min":{"1min":0.36199999999999999,"5min":0.309,"15min":0.25900000000000001},"max":{"1min":4.1669999999999998,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.2519999999999998,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":3.165,"5min":3.165,"15min":4.851},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.254,"5min":0.23400000000000001,"15min":0.22500000000000001},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":5.0590000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.41399999999999998,"15min":0.44900000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.23300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":5.2679999999999998},"last":0.375}]},{"osd":5,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.878,"5min":1.3089999999999999,"15min":3.5760000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.51800000000000002,"15min":0.54500000000000004},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":6.4790000000000001},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":1.153,"5min":1.2529999999999999,"15min":4.157},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.29299999999999998},"max":{"1min":2.403,"5min":2.403,"15min":5.0880000000000001},"last":0.434}]}]},{"osd":6,"up_from":13,"seq":55834575475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101820,"kb_used_data":69876,"kb_used_omap":1,"kb_used_meta":1031934,"kb_avail":93270020,"statfs":{"total":96636764160,"available":95508500480,"internally_reserved":0,"allocated":71553024,"data_stored":71391599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1056700683},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.47299999999999998,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.188},"max":{"1min":2.899,"5min":2.899,"15min":17.885999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.23000000000000001},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.72199999999999998}]},{"osd":1,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.52000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.216},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":7.226},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":1.224,"15min":7.1699999999999999},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.6189999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.26800000000000002},"max":{"1min":3.282,"5min":3.282,"15min":7.2489999999999997},"last":0.75}]},{"osd":3,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":1.2470000000000001,"15min":7.1470000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.61799999999999999,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.29899999999999999},"max":{"1min":6.1660000000000004,"5min":6.1660000000000004,"15min":6.1660000000000004},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.468,"5min":2.468,"15min":7.3330000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.49199999999999999,"15min":0.45300000000000001},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":6.0229999999999997,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":8.234,"5min":8.234,"15min":8.234},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48799999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.90600000000000003,"5min":1.3340000000000001,"15min":7.5380000000000003},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.54100000000000004,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":7.2039999999999997},"last":0.622}]}]},{"osd":5,"up_from":13,"seq":55834575475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1151736,"kb_used_data":69872,"kb_used_omap":1,"kb_used_meta":1081854,"kb_avail":93220104,"statfs":{"total":96636764160,"available":95457386496,"internally_reserved":0,"allocated":71548928,"data_stored":71391017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1107818763},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.54300000000000004},"min":{"1min":0.26200000000000001,"5min":0.223,"15min":0.19400000000000001},"max":{"1min":0.66100000000000003,"5min":3.2149999999999999,"15min":7.0990000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.53000000000000003,"15min":0.57299999999999995},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.21099999999999999},"max":{"1min":0.73799999999999999,"5min":5.7359999999999998,"15min":12.281000000000001},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.68799999999999994,"15min":0.57599999999999996},"min":{"1min":0.245,"5min":0.245,"15min":0.20599999999999999},"max":{"1min":0.63100000000000001,"5min":14.952,"15min":14.952},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.52200000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.247},"max":{"1min":0.745,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.75800000000000001,"15min":0.61599999999999999},"min":{"1min":0.29099999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":18.623000000000001,"15min":18.623000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.96099999999999997,"15min":0.67300000000000004},"min":{"1min":0.32200000000000001,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":30.004999999999999,"15min":30.004999999999999},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28000000000000003},"max":{"1min":0.69599999999999995,"5min":3.6600000000000001,"15min":9.2309999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":0.82699999999999996,"15min":7.3780000000000001},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.59899999999999998,"15min":0.50700000000000001},"min":{"1min":0.23599999999999999,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.51900000000000002,"5min":12.414,"15min":12.414},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.50800000000000001,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.66900000000000004,"5min":3.1970000000000001,"15min":8.3339999999999996},"last":0.31}]},{"osd":6,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45100000000000001,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.245,"15min":0.22},"max":{"1min":0.64000000000000001,"5min":1.905,"15min":7.4000000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.621,"15min":0.55400000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.24299999999999999},"max":{"1min":0.63200000000000001,"5min":10.278,"15min":10.278},"last":0.378}]},{"osd":7,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":0.68500000000000005,"5min":3.262,"15min":7.1619999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54700000000000004,"15min":0.56000000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":3.2909999999999999,"15min":7.3310000000000004},"last":0.39900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575476,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":965948,"kb_used_data":42292,"kb_used_omap":1,"kb_used_meta":923646,"kb_avail":93405892,"statfs":{"total":96636764160,"available":95647633408,"internally_reserved":0,"allocated":43307008,"data_stored":43137601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":945813771},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.502,"15min":0.56200000000000006},"min":{"1min":0.41099999999999998,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":1.1319999999999999,"5min":2.23,"15min":15.013},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.20499999999999999},"max":{"1min":1.3120000000000001,"5min":2.1139999999999999,"15min":5.6879999999999997},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.254},"max":{"1min":1.3500000000000001,"5min":1.806,"15min":4.6479999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.48799999999999999,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":4.0469999999999997},"last":0.749}]},{"osd":2,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.29099999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":5.5540000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34399999999999997,"15min":0.29599999999999999},"max":{"1min":1.379,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.627}]},{"osd":3,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":9.9800000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.899},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.441,"15min":0.41899999999999998},"min":{"1min":0.25,"5min":0.219,"15min":0.219},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.0470000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.22700000000000001},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.245},"max":{"1min":2.21,"5min":2.21,"15min":3.8650000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.495,"15min":0.46899999999999997},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.24299999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.501,"15min":0.505},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23400000000000001},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":8.7639999999999993},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52700000000000002,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":3.7850000000000001},"last":0.60999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1135420,"kb_used_data":57588,"kb_used_omap":1,"kb_used_meta":1077822,"kb_avail":93236420,"statfs":{"total":96636764160,"available":95474094080,"internally_reserved":0,"allocated":58970112,"data_stored":58808687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1103689995},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.17000000000000001},"max":{"1min":8.3350000000000009,"5min":8.3350000000000009,"15min":14.695},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.41699999999999998,"15min":0.52100000000000002},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.186},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":8.9649999999999999},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.41299999999999998,"15min":0.46000000000000002},"min":{"1min":0.22,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":5.5289999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.55400000000000005,"15min":0.59199999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.20100000000000001},"max":{"1min":1.508,"5min":6.1740000000000004,"15min":14.898},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46300000000000002,"15min":0.56799999999999995},"min":{"1min":0.26300000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61599999999999999,"5min":3.7869999999999999,"15min":9.5150000000000006},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.28899999999999998,"5min":0.25800000000000001,"15min":0.23100000000000001},"max":{"1min":2.2090000000000001,"5min":2.9430000000000001,"15min":5.2859999999999996},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.74399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":0.84299999999999997,"5min":1.5229999999999999,"15min":14.071999999999999},"last":1.075},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.60699999999999998,"15min":0.69899999999999995},"min":{"1min":0.40000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":14.109},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64400000000000002,"15min":0.68100000000000005},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.314},"max":{"1min":3.0209999999999999,"5min":5.1609999999999996,"15min":11.443},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.2589999999999999,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.27800000000000002},"max":{"1min":8.3659999999999997,"5min":8.3659999999999997,"15min":14.550000000000001},"last":1.054}]},{"osd":6,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65600000000000003,"15min":0.751},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.29299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2539999999999996,"15min":9.7200000000000006},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.65700000000000003,"15min":0.70699999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35599999999999998},"max":{"1min":2.6360000000000001,"5min":2.8170000000000002,"15min":15.339},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":8.2799999999999994,"5min":8.2799999999999994,"15min":14.726000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64200000000000002,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.33700000000000002},"max":{"1min":1.1180000000000001,"5min":2.9489999999999998,"15min":9.5079999999999991},"last":1.101}]}]},{"osd":2,"up_from":13,"seq":55834575476,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649796,"kb_used_data":45244,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":93722044,"statfs":{"total":96636764160,"available":95971373056,"internally_reserved":0,"allocated":46329856,"data_stored":46172026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":619051275},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.58199999999999996,"15min":0.497},"min":{"1min":0.217,"5min":0.185,"15min":0.182},"max":{"1min":6.3840000000000003,"5min":7.008,"15min":10.869},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.56799999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":3.8170000000000002,"5min":7.7699999999999996,"15min":12.317},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.59699999999999998,"15min":0.49099999999999999},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":6.4249999999999998,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.20100000000000001},"max":{"1min":3.8769999999999998,"5min":7.8090000000000002,"15min":9.5429999999999993},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.63100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":2.5419999999999998,"5min":13.130000000000001,"15min":27.355},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.214,"5min":0.64700000000000002,"15min":0.62},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27500000000000002},"max":{"1min":7.5910000000000002,"5min":7.5910000000000002,"15min":12.451000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":0.90900000000000003,"15min":0.77200000000000002},"min":{"1min":0.46400000000000002,"5min":0.32100000000000001,"15min":0.28899999999999998},"max":{"1min":7.6239999999999997,"5min":8.0440000000000005,"15min":12.478999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.3289999999999997,"5min":7.9409999999999998,"15min":12.391999999999999},"last":0.79900000000000004}]},{"osd":5,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.90200000000000002,"15min":0.76700000000000002},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":7.4089999999999998,"5min":13.318,"15min":20.84},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.5549999999999999,"5min":0.79300000000000004,"15min":0.73699999999999999},"min":{"1min":0.40799999999999997,"5min":0.35799999999999998,"15min":0.32000000000000001},"max":{"1min":12.319000000000001,"5min":12.319000000000001,"15min":12.319000000000001},"last":0.76100000000000001}]},{"osd":6,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.65400000000000003,"15min":0.66100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34300000000000003},"max":{"1min":2.6429999999999998,"5min":5.0209999999999999,"15min":6.2560000000000002},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.05,"5min":0.84899999999999998,"15min":0.68700000000000006},"min":{"1min":0.40100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.4420000000000002,"5min":13.177,"15min":13.177},"last":0.628}]},{"osd":7,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.74299999999999999,"15min":0.71899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.32600000000000001},"max":{"1min":3.8490000000000002,"5min":4.9800000000000004,"15min":6.2309999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.504,"5min":3.5830000000000002,"15min":6.4279999999999999},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575475,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1031244,"kb_used_data":57540,"kb_used_omap":1,"kb_used_meta":973694,"kb_avail":93340596,"statfs":{"total":96636764160,"available":95580770304,"internally_reserved":0,"allocated":58920960,"data_stored":58755098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":997062923},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.186},"max":{"1min":1.7090000000000001,"5min":3.625,"15min":10.891999999999999},"last":0.307},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46999999999999997,"15min":0.50700000000000001},"min":{"1min":0.24399999999999999,"5min":0.219,"15min":0.19700000000000001},"max":{"1min":1.341,"5min":6.5339999999999998,"15min":9.2249999999999996},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.49199999999999999},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.20999999999999999},"max":{"1min":2.6589999999999998,"5min":13.553000000000001,"15min":13.553000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50700000000000001,"15min":0.53400000000000003},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.23300000000000001},"max":{"1min":3.008,"5min":6.6050000000000004,"15min":7.1180000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.307,"5min":0.251,"15min":0.22900000000000001},"max":{"1min":1.3120000000000001,"5min":10.236000000000001,"15min":24.879000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.53800000000000003,"15min":0.57199999999999995},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.22900000000000001},"max":{"1min":10.433,"5min":10.433,"15min":20.995000000000001},"last":0.28499999999999998}]},{"osd":4,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":1.748,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":2.9390000000000001,"5min":8.4420000000000002,"15min":8.4420000000000002},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.621,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.3940000000000001,"5min":6.3940000000000001,"15min":8.6419999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80200000000000005,"15min":0.70299999999999996},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.8100000000000001,"5min":16.943999999999999,"15min":16.943999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.32100000000000001},"max":{"1min":2.379,"5min":2.379,"15min":7.468},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.33600000000000002},"max":{"1min":2.3290000000000002,"5min":3.4950000000000001,"15min":5.5030000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":2.9780000000000002,"5min":6.5720000000000001,"15min":9.3209999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.36099999999999999},"max":{"1min":1.52,"5min":3.8959999999999999,"15min":10.346},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":871104,"kb_used_data":49976,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":93500736,"statfs":{"total":96636764160,"available":95744753664,"internally_reserved":0,"allocated":51175424,"data_stored":51009794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":840825099},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.45500000000000002},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.17499999999999999},"max":{"1min":2.8149999999999999,"5min":12.734,"15min":12.734},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52000000000000002,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":3.976,"5min":9.3620000000000001,"15min":9.3620000000000001},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.843,"5min":15.375,"15min":15.375},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.48099999999999998},"min":{"1min":0.28399999999999997,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.008,"5min":9.2959999999999994,"15min":9.2959999999999994},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.20699999999999999},"max":{"1min":2.7040000000000002,"5min":8.7230000000000008,"15min":8.7230000000000008},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":2.9169999999999998,"5min":10.928000000000001,"15min":12.864000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0350000000000001,"5min":9.2319999999999993,"15min":9.2319999999999993},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":3.7930000000000001,"15min":7.5499999999999998},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.61899999999999999},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":3.0790000000000002,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.747},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.68700000000000006},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"max":{"1min":2.8660000000000001,"5min":10.135999999999999,"15min":11.068},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.69699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.8959999999999999,"5min":12.77,"15min":12.77},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":4.0579999999999998,"5min":9.391,"15min":9.391},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73299999999999998,"15min":0.67900000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.0830000000000002,"5min":9.3290000000000006,"15min":9.3290000000000006},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.71499999999999997,"15min":0.65400000000000003},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.33300000000000002},"max":{"1min":0.78900000000000003,"5min":7.8739999999999997,"15min":7.8739999999999997},"last":0.84599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58679296,"data_stored":58667339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":46088192,"data_stored":46084267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43065344,"data_stored":43049842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":71307264,"data_stored":71303258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":71311360,"data_stored":71303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114360320,"data_stored":114353466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T22:05:53.090 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.090 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.090 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.090 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.090 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.090 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.091 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.092 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.093 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.094 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.095 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.096 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.096 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.096 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-03-14T21:14:01.634028+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=1, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.096 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.096 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:05:53.096 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-14T22:05:53.718 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:05:53.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:05:53.723 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:13.098 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T22:06:13.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.218+0000 7f5aede38640 1 Processor -- start 2024-03-14T22:06:13.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.218+0000 7f5aede38640 1 -- start start 2024-03-14T22:06:13.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5aede38640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ae80baef0 0x7f5ae81480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:13.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5aede38640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ae80bc940 0x7f5ae8155710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5aede38640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae806a6e0 con 0x7f5ae806b620 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5aede38640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae806ad10 con 0x7f5ae80baef0 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5aede38640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae80bc2b0 con 0x7f5ae80bc940 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae77fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ae80baef0 0x7f5ae81480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae77fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ae80baef0 0x7f5ae81480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42034/0 (socket says 172.21.15.49:42034) 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2892917547 learned_addr learned my addr 172.21.15.49:0/2892917547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae7fff640 1 --2- 172.21.15.49:0/2892917547 >> v2:172.21.15.97:3300/0 conn(0x7f5ae80bc940 0x7f5ae8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2892917547 >> v2:172.21.15.97:3300/0 conn(0x7f5ae80bc940 msgr2=0x7f5ae8155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae77fe640 1 --2- 172.21.15.49:0/2892917547 >> v2:172.21.15.97:3300/0 conn(0x7f5ae80bc940 0x7f5ae8155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2892917547 >> v1:172.21.15.97:6789/0 conn(0x7f5ae806b620 legacy=0x7f5ae80bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.219+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2892917547 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae806bab0 con 0x7f5ae80baef0 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.220+0000 7f5ae77fe640 1 --2- 172.21.15.49:0/2892917547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae81480b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5ae814f480 tx=0x7f5ad802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ada88f99743e377 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.220+0000 7f5ae67fc640 1 -- 172.21.15.49:0/2892917547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8031870 con 0x7f5ae80baef0 2024-03-14T22:06:13.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.220+0000 7f5ae67fc640 1 -- 172.21.15.49:0/2892917547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad80319d0 con 0x7f5ae80baef0 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.220+0000 7f5ae67fc640 1 -- 172.21.15.49:0/2892917547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8031d60 con 0x7f5ae80baef0 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 -- 172.21.15.49:0/2892917547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 msgr2=0x7f5ae81480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 --2- 172.21.15.49:0/2892917547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae81480b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5ae814f480 tx=0x7f5ad802b910 comp rx=0 tx=0).stop 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 -- 172.21.15.49:0/2892917547 shutdown_connections 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 --2- 172.21.15.49:0/2892917547 >> v2:172.21.15.97:3300/0 conn(0x7f5ae80bc940 0x7f5ae8155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 --2- 172.21.15.49:0/2892917547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae81480b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 -- 172.21.15.49:0/2892917547 >> 172.21.15.49:0/2892917547 conn(0x7f5ae80bcd80 msgr2=0x7f5ae80bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 -- 172.21.15.49:0/2892917547 shutdown_connections 2024-03-14T22:06:13.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.221+0000 7f5aede38640 1 -- 172.21.15.49:0/2892917547 wait complete. 2024-03-14T22:06:13.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.222+0000 7f5aede38640 1 Processor -- start 2024-03-14T22:06:13.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.222+0000 7f5aede38640 1 -- start start 2024-03-14T22:06:13.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5aede38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae80a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:13.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5aede38640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ae80bc940 0x7f5ae8145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:13.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5aede38640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80b6070 0x7f5ae81e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5aede38640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae809eec0 con 0x7f5ae80baef0 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5aede38640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81dfa50 con 0x7f5ae80b6070 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5aede38640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81e02f0 con 0x7f5ae80bc940 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae80a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae80a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42042/0 (socket says 172.21.15.49:42042) 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2447610817 learned_addr learned my addr 172.21.15.49:0/2447610817 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae7fff640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80b6070 0x7f5ae81e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae6ffd640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ae80bc940 0x7f5ae8145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ae80bc940 msgr2=0x7f5ae8145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ae80bc940 0x7f5ae8145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80b6070 msgr2=0x7f5ae81e2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80b6070 0x7f5ae81e2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 -- 172.21.15.49:0/2447610817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80094d0 con 0x7f5ae80baef0 2024-03-14T22:06:13.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5ae77fe640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae80a13d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:13.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.223+0000 7f5acffff640 1 -- 172.21.15.49:0/2447610817 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad803ac60 con 0x7f5ae80baef0 2024-03-14T22:06:13.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.224+0000 7f5acffff640 1 -- 172.21.15.49:0/2447610817 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad803adc0 con 0x7f5ae80baef0 2024-03-14T22:06:13.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.224+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8146860 con 0x7f5ae80baef0 2024-03-14T22:06:13.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.224+0000 7f5acffff640 1 -- 172.21.15.49:0/2447610817 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80445f0 con 0x7f5ae80baef0 2024-03-14T22:06:13.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.224+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81dff80 con 0x7f5ae80baef0 2024-03-14T22:06:13.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.225+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae80666a0 con 0x7f5ae80baef0 2024-03-14T22:06:13.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.225+0000 7f5acffff640 1 -- 172.21.15.49:0/2447610817 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5ad8044750 con 0x7f5ae80baef0 2024-03-14T22:06:13.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.225+0000 7f5acffff640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f5ad406a7a0 0x7f5ad406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:13.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.226+0000 7f5acffff640 1 -- 172.21.15.49:0/2447610817 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5ad802d040 con 0x7f5ae80baef0 2024-03-14T22:06:13.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.226+0000 7f5ae6ffd640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f5ad406a7a0 0x7f5ad406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:13.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.226+0000 7f5ae6ffd640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f5ad406a7a0 0x7f5ad406cc60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8148f90 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:13.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.230+0000 7f5acffff640 1 -- 172.21.15.49:0/2447610817 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5ad8076870 con 0x7f5ae80baef0 2024-03-14T22:06:13.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.407+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ae80bb2d0 con 0x7f5ad406a7a0 2024-03-14T22:06:13.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.414+0000 7f5acffff640 1 -- 172.21.15.49:0/2447610817 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293656 (secure 0 0 0) 0x7f5adc0050d0 con 0x7f5ad406a7a0 2024-03-14T22:06:13.416 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:06:13.418 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T22:06:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f5ad406a7a0 msgr2=0x7f5ad406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f5ad406a7a0 0x7f5ad406cc60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8148f90 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2024-03-14T22:06:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 msgr2=0x7f5ae80a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae80a13d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad80046c0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 shutdown_connections 2024-03-14T22:06:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f5ad406a7a0 0x7f5ad406cc60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80b6070 0x7f5ae81e2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.417+0000 7f5aede38640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ae80bc940 0x7f5ae8145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.418+0000 7f5aede38640 1 --2- 172.21.15.49:0/2447610817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae80baef0 0x7f5ae80a13d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.418+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 >> 172.21.15.49:0/2447610817 conn(0x7f5ae80bcd80 msgr2=0x7f5ae8072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.418+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 shutdown_connections 2024-03-14T22:06:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:13.418+0000 7f5aede38640 1 -- 172.21.15.49:0/2447610817 wait complete. 2024-03-14T22:06:13.427 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":1587,"stamp":"2024-03-14T22:06:11.600155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257497981,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":4402,"num_read_kb":112073,"num_write":843876,"num_write_kb":9407945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23590,"num_omap_keys":34,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896084,"ondisk_log_size":896084,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8159732,"kb_used_data":504884,"kb_used_omap":13,"kb_used_meta":7654770,"kb_avail":746814988,"statfs":{"total":773094113280,"available":764738547712,"internally_reserved":0,"allocated":517001216,"data_stored":515697415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":7838484568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003024"},"pg_stats":[{"pgid":"3.3f","version":"23'10331","reported_seq":10375,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:32.411713+0000","last_change":"2024-03-14T21:14:03.030580+0000","last_active":"2024-03-14T22:05:32.411713+0000","last_peered":"2024-03-14T22:05:32.411713+0000","last_clean":"2024-03-14T22:05:32.411713+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T22:05:32.411713+0000","last_undegraded":"2024-03-14T22:05:32.411713+0000","last_fullsized":"2024-03-14T22:05:32.411713+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8785,"num_write_kb":106962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'9098","reported_seq":9160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:04.441648+0000","last_change":"2024-03-14T22:06:04.441648+0000","last_active":"2024-03-14T22:06:04.441648+0000","last_peered":"2024-03-14T22:06:04.441648+0000","last_clean":"2024-03-14T22:06:04.441648+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T22:06:04.441648+0000","last_undegraded":"2024-03-14T22:06:04.441648+0000","last_fullsized":"2024-03-14T22:06:04.441648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9098","last_scrub_stamp":"2024-03-14T22:06:04.441479+0000","last_deep_scrub":"23'9098","last_deep_scrub_stamp":"2024-03-14T22:06:04.441479+0000","last_clean_scrub_stamp":"2024-03-14T22:06:04.441479+0000","objects_scrubbed":0,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:23.176730+0000","scrub_duration":0.011687632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":7584,"num_write_kb":103945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'11564","reported_seq":11618,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:08.135625+0000","last_change":"2024-03-14T22:06:08.135625+0000","last_active":"2024-03-14T22:06:08.135625+0000","last_peered":"2024-03-14T22:06:08.135625+0000","last_clean":"2024-03-14T22:06:08.135625+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T22:06:08.135625+0000","last_undegraded":"2024-03-14T22:06:08.135625+0000","last_fullsized":"2024-03-14T22:06:08.135625+0000","mapping_epoch":19,"log_start":"23'1500","ondisk_log_start":"23'1500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11564","last_scrub_stamp":"2024-03-14T22:06:08.135455+0000","last_deep_scrub":"23'11564","last_deep_scrub_stamp":"2024-03-14T22:06:08.135455+0000","last_clean_scrub_stamp":"2024-03-14T22:06:08.135455+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:26.870706+0000","scrub_duration":0.011002987000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1274,"num_write":9914,"num_write_kb":127294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'10102","reported_seq":10160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:09.658703+0000","last_change":"2024-03-14T22:06:09.658830+0000","last_active":"2024-03-14T22:06:09.658703+0000","last_peered":"2024-03-14T22:06:09.658703+0000","last_clean":"2024-03-14T22:06:09.658703+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T22:06:09.658703+0000","last_undegraded":"2024-03-14T22:06:09.658703+0000","last_fullsized":"2024-03-14T22:06:09.658703+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10102","last_scrub_stamp":"2024-03-14T22:06:09.658681+0000","last_deep_scrub":"23'10102","last_deep_scrub_stamp":"2024-03-14T22:06:09.658681+0000","last_clean_scrub_stamp":"2024-03-14T22:06:09.658681+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:25:59.909681+0000","scrub_duration":0.014752894000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8407,"num_write_kb":110879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'9608","reported_seq":9643,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.748202+0000","last_change":"2024-03-14T21:14:03.033493+0000","last_active":"2024-03-14T22:05:25.748202+0000","last_peered":"2024-03-14T22:05:25.748202+0000","last_clean":"2024-03-14T22:05:25.748202+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T22:05:25.748202+0000","last_undegraded":"2024-03-14T22:05:25.748202+0000","last_fullsized":"2024-03-14T22:05:25.748202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9608,"log_dups_size":0,"ondisk_log_size":9608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8008,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'9645","reported_seq":9686,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.885051+0000","last_change":"2024-03-14T21:14:03.032386+0000","last_active":"2024-03-14T22:05:25.885051+0000","last_peered":"2024-03-14T22:05:25.885051+0000","last_clean":"2024-03-14T22:05:25.885051+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T22:05:25.885051+0000","last_undegraded":"2024-03-14T22:05:25.885051+0000","last_fullsized":"2024-03-14T22:05:25.885051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9645,"log_dups_size":0,"ondisk_log_size":9645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":8054,"num_write_kb":114907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10037","reported_seq":10074,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.757404+0000","last_change":"2024-03-14T21:14:03.030605+0000","last_active":"2024-03-14T22:05:25.757404+0000","last_peered":"2024-03-14T22:05:25.757404+0000","last_clean":"2024-03-14T22:05:25.757404+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T22:05:25.757404+0000","last_undegraded":"2024-03-14T22:05:25.757404+0000","last_fullsized":"2024-03-14T22:05:25.757404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":8423,"num_write_kb":104789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'11196","reported_seq":11237,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.859079+0000","last_change":"2024-03-14T21:14:03.039365+0000","last_active":"2024-03-14T22:05:25.859079+0000","last_peered":"2024-03-14T22:05:25.859079+0000","last_clean":"2024-03-14T22:05:25.859079+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T22:05:25.859079+0000","last_undegraded":"2024-03-14T22:05:25.859079+0000","last_fullsized":"2024-03-14T22:05:25.859079+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":9605,"num_write_kb":125652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'9017","reported_seq":9055,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.837419+0000","last_change":"2024-03-14T21:14:03.030748+0000","last_active":"2024-03-14T22:05:25.837419+0000","last_peered":"2024-03-14T22:05:25.837419+0000","last_clean":"2024-03-14T22:05:25.837419+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T22:05:25.837419+0000","last_undegraded":"2024-03-14T22:05:25.837419+0000","last_fullsized":"2024-03-14T22:05:25.837419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9017,"log_dups_size":0,"ondisk_log_size":9017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7381,"num_write_kb":104568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'9539","reported_seq":9576,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.843247+0000","last_change":"2024-03-14T21:14:03.034825+0000","last_active":"2024-03-14T22:05:25.843247+0000","last_peered":"2024-03-14T22:05:25.843247+0000","last_clean":"2024-03-14T22:05:25.843247+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T22:05:25.843247+0000","last_undegraded":"2024-03-14T22:05:25.843247+0000","last_fullsized":"2024-03-14T22:05:25.843247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7843,"num_write_kb":113016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'9229","reported_seq":9282,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.535716+0000","last_change":"2024-03-14T22:06:01.535842+0000","last_active":"2024-03-14T22:06:01.535716+0000","last_peered":"2024-03-14T22:06:01.535716+0000","last_clean":"2024-03-14T22:06:01.535716+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T22:06:01.535716+0000","last_undegraded":"2024-03-14T22:06:01.535716+0000","last_fullsized":"2024-03-14T22:06:01.535716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9229","last_scrub_stamp":"2024-03-14T22:06:01.535693+0000","last_deep_scrub":"23'9229","last_deep_scrub_stamp":"2024-03-14T22:06:01.535693+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.535693+0000","objects_scrubbed":0,"log_size":9229,"log_dups_size":0,"ondisk_log_size":9229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:00.834979+0000","scrub_duration":0.011388333,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7677,"num_write_kb":104837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'10311","reported_seq":10358,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:36.777685+0000","last_change":"2024-03-14T22:05:36.777822+0000","last_active":"2024-03-14T22:05:36.777685+0000","last_peered":"2024-03-14T22:05:36.777685+0000","last_clean":"2024-03-14T22:05:36.777685+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T22:05:36.777685+0000","last_undegraded":"2024-03-14T22:05:36.777685+0000","last_fullsized":"2024-03-14T22:05:36.777685+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10311","last_scrub_stamp":"2024-03-14T22:05:36.777662+0000","last_deep_scrub":"23'10311","last_deep_scrub_stamp":"2024-03-14T22:05:36.777662+0000","last_clean_scrub_stamp":"2024-03-14T22:05:36.777662+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:58:06.567940+0000","scrub_duration":0.019761048999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8730,"num_write_kb":105734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'8864","reported_seq":8914,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:59.996609+0000","last_change":"2024-03-14T22:05:59.996747+0000","last_active":"2024-03-14T22:05:59.996609+0000","last_peered":"2024-03-14T22:05:59.996609+0000","last_clean":"2024-03-14T22:05:59.996609+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T22:05:59.996609+0000","last_undegraded":"2024-03-14T22:05:59.996609+0000","last_fullsized":"2024-03-14T22:05:59.996609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8864","last_scrub_stamp":"2024-03-14T22:05:59.996584+0000","last_deep_scrub":"23'8864","last_deep_scrub_stamp":"2024-03-14T22:05:59.996584+0000","last_clean_scrub_stamp":"2024-03-14T22:05:59.996584+0000","objects_scrubbed":0,"log_size":8864,"log_dups_size":0,"ondisk_log_size":8864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:10.781627+0000","scrub_duration":0.0095402560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":7264,"num_write_kb":95419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'10416","reported_seq":10468,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:09.178102+0000","last_change":"2024-03-14T22:06:09.178225+0000","last_active":"2024-03-14T22:06:09.178102+0000","last_peered":"2024-03-14T22:06:09.178102+0000","last_clean":"2024-03-14T22:06:09.178102+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T22:06:09.178102+0000","last_undegraded":"2024-03-14T22:06:09.178102+0000","last_fullsized":"2024-03-14T22:06:09.178102+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10416","last_scrub_stamp":"2024-03-14T22:06:09.178081+0000","last_deep_scrub":"23'10416","last_deep_scrub_stamp":"2024-03-14T22:06:09.178081+0000","last_clean_scrub_stamp":"2024-03-14T22:06:09.178081+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:39.997111+0000","scrub_duration":0.014718489,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8793,"num_write_kb":111405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'9704","reported_seq":9754,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:52.497668+0000","last_change":"2024-03-14T22:05:52.497829+0000","last_active":"2024-03-14T22:05:52.497668+0000","last_peered":"2024-03-14T22:05:52.497668+0000","last_clean":"2024-03-14T22:05:52.497668+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T22:05:52.497668+0000","last_undegraded":"2024-03-14T22:05:52.497668+0000","last_fullsized":"2024-03-14T22:05:52.497668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9704","last_scrub_stamp":"2024-03-14T22:05:52.497623+0000","last_deep_scrub":"23'9704","last_deep_scrub_stamp":"2024-03-14T22:05:52.497623+0000","last_clean_scrub_stamp":"2024-03-14T22:05:52.497623+0000","objects_scrubbed":0,"log_size":9704,"log_dups_size":0,"ondisk_log_size":9704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:42.520325+0000","scrub_duration":0.011931638,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8012,"num_write_kb":95820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'9661","reported_seq":9721,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.646033+0000","last_change":"2024-03-14T22:06:01.646136+0000","last_active":"2024-03-14T22:06:01.646033+0000","last_peered":"2024-03-14T22:06:01.646033+0000","last_clean":"2024-03-14T22:06:01.646033+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T22:06:01.646033+0000","last_undegraded":"2024-03-14T22:06:01.646033+0000","last_fullsized":"2024-03-14T22:06:01.646033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9661","last_scrub_stamp":"2024-03-14T22:06:01.646010+0000","last_deep_scrub":"23'9661","last_deep_scrub_stamp":"2024-03-14T22:06:01.646010+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.646010+0000","objects_scrubbed":0,"log_size":9661,"log_dups_size":0,"ondisk_log_size":9661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:19:19.857867+0000","scrub_duration":0.015379894999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3572,"num_write":8076,"num_write_kb":107857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8662","reported_seq":8712,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:53.158066+0000","last_change":"2024-03-14T22:05:53.158066+0000","last_active":"2024-03-14T22:05:53.158066+0000","last_peered":"2024-03-14T22:05:53.158066+0000","last_clean":"2024-03-14T22:05:53.158066+0000","last_became_active":"2024-03-14T21:14:44.858906+0000","last_became_peered":"2024-03-14T21:14:44.858906+0000","last_unstale":"2024-03-14T22:05:53.158066+0000","last_undegraded":"2024-03-14T22:05:53.158066+0000","last_fullsized":"2024-03-14T22:05:53.158066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8662","last_scrub_stamp":"2024-03-14T22:05:53.157928+0000","last_deep_scrub":"23'8662","last_deep_scrub_stamp":"2024-03-14T22:05:53.157928+0000","last_clean_scrub_stamp":"2024-03-14T22:05:53.157928+0000","objects_scrubbed":0,"log_size":8662,"log_dups_size":0,"ondisk_log_size":8662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:52.457213+0000","scrub_duration":0.0057217580000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1528,"num_write":7020,"num_write_kb":102847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10199","reported_seq":10247,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:46.076003+0000","last_change":"2024-03-14T22:05:46.076125+0000","last_active":"2024-03-14T22:05:46.076003+0000","last_peered":"2024-03-14T22:05:46.076003+0000","last_clean":"2024-03-14T22:05:46.076003+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T22:05:46.076003+0000","last_undegraded":"2024-03-14T22:05:46.076003+0000","last_fullsized":"2024-03-14T22:05:46.076003+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10199","last_scrub_stamp":"2024-03-14T22:05:46.075978+0000","last_deep_scrub":"23'10199","last_deep_scrub_stamp":"2024-03-14T22:05:46.075978+0000","last_clean_scrub_stamp":"2024-03-14T22:05:46.075978+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:28.104989+0000","scrub_duration":0.011461096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8552,"num_write_kb":105487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'10521","reported_seq":10568,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.528915+0000","last_change":"2024-03-14T22:06:01.528915+0000","last_active":"2024-03-14T22:06:01.528915+0000","last_peered":"2024-03-14T22:06:01.528915+0000","last_clean":"2024-03-14T22:06:01.528915+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T22:06:01.528915+0000","last_undegraded":"2024-03-14T22:06:01.528915+0000","last_fullsized":"2024-03-14T22:06:01.528915+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10521","last_scrub_stamp":"2024-03-14T22:06:01.528722+0000","last_deep_scrub":"23'10521","last_deep_scrub_stamp":"2024-03-14T22:06:01.528722+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.528722+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:43:24.576763+0000","scrub_duration":0.012349428000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":118219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'10900","reported_seq":10951,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:56.603066+0000","last_change":"2024-03-14T22:05:56.603174+0000","last_active":"2024-03-14T22:05:56.603066+0000","last_peered":"2024-03-14T22:05:56.603066+0000","last_clean":"2024-03-14T22:05:56.603066+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T22:05:56.603066+0000","last_undegraded":"2024-03-14T22:05:56.603066+0000","last_fullsized":"2024-03-14T22:05:56.603066+0000","mapping_epoch":19,"log_start":"23'800","ondisk_log_start":"23'800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10900","last_scrub_stamp":"2024-03-14T22:05:56.603044+0000","last_deep_scrub":"23'10900","last_deep_scrub_stamp":"2024-03-14T22:05:56.603044+0000","last_clean_scrub_stamp":"2024-03-14T22:05:56.603044+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:46.625746+0000","scrub_duration":0.013839676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":9315,"num_write_kb":117297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'10962","reported_seq":11022,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.181353+0000","last_change":"2024-03-14T22:06:05.181507+0000","last_active":"2024-03-14T22:06:05.181353+0000","last_peered":"2024-03-14T22:06:05.181353+0000","last_clean":"2024-03-14T22:06:05.181353+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T22:06:05.181353+0000","last_undegraded":"2024-03-14T22:06:05.181353+0000","last_fullsized":"2024-03-14T22:06:05.181353+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10962","last_scrub_stamp":"2024-03-14T22:06:05.181330+0000","last_deep_scrub":"23'10962","last_deep_scrub_stamp":"2024-03-14T22:06:05.181330+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.181330+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:15.966373+0000","scrub_duration":0.011570298999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3050,"num_write":9338,"num_write_kb":109552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10723","reported_seq":10772,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:08.662770+0000","last_change":"2024-03-14T22:06:08.662902+0000","last_active":"2024-03-14T22:06:08.662770+0000","last_peered":"2024-03-14T22:06:08.662770+0000","last_clean":"2024-03-14T22:06:08.662770+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T22:06:08.662770+0000","last_undegraded":"2024-03-14T22:06:08.662770+0000","last_fullsized":"2024-03-14T22:06:08.662770+0000","mapping_epoch":19,"log_start":"23'700","ondisk_log_start":"23'700","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10723","last_scrub_stamp":"2024-03-14T22:06:08.662746+0000","last_deep_scrub":"23'10723","last_deep_scrub_stamp":"2024-03-14T22:06:08.662746+0000","last_clean_scrub_stamp":"2024-03-14T22:06:08.662746+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:27:41.265598+0000","scrub_duration":0.011816327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":9066,"num_write_kb":118743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'10565","reported_seq":10614,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:03.942607+0000","last_change":"2024-03-14T22:06:03.942724+0000","last_active":"2024-03-14T22:06:03.942607+0000","last_peered":"2024-03-14T22:06:03.942607+0000","last_clean":"2024-03-14T22:06:03.942607+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T22:06:03.942607+0000","last_undegraded":"2024-03-14T22:06:03.942607+0000","last_fullsized":"2024-03-14T22:06:03.942607+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10565","last_scrub_stamp":"2024-03-14T22:06:03.942584+0000","last_deep_scrub":"23'10565","last_deep_scrub_stamp":"2024-03-14T22:06:03.942584+0000","last_clean_scrub_stamp":"2024-03-14T22:06:03.942584+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:34.761614+0000","scrub_duration":0.013751275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8950,"num_write_kb":109690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'11180","reported_seq":11233,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:49.737590+0000","last_change":"2024-03-14T22:05:49.737865+0000","last_active":"2024-03-14T22:05:49.737590+0000","last_peered":"2024-03-14T22:05:49.737590+0000","last_clean":"2024-03-14T22:05:49.737590+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T22:05:49.737590+0000","last_undegraded":"2024-03-14T22:05:49.737590+0000","last_fullsized":"2024-03-14T22:05:49.737590+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11180","last_scrub_stamp":"2024-03-14T22:05:49.737563+0000","last_deep_scrub":"23'11180","last_deep_scrub_stamp":"2024-03-14T22:05:49.737563+0000","last_clean_scrub_stamp":"2024-03-14T22:05:49.737563+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:00.522606+0000","scrub_duration":0.013646538999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1530,"num_write":9571,"num_write_kb":127549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'10195","reported_seq":10245,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:50.983228+0000","last_change":"2024-03-14T22:05:50.983228+0000","last_active":"2024-03-14T22:05:50.983228+0000","last_peered":"2024-03-14T22:05:50.983228+0000","last_clean":"2024-03-14T22:05:50.983228+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T22:05:50.983228+0000","last_undegraded":"2024-03-14T22:05:50.983228+0000","last_fullsized":"2024-03-14T22:05:50.983228+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10195","last_scrub_stamp":"2024-03-14T22:05:50.983096+0000","last_deep_scrub":"23'10195","last_deep_scrub_stamp":"2024-03-14T22:05:50.983096+0000","last_clean_scrub_stamp":"2024-03-14T22:05:50.983096+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:41.005798+0000","scrub_duration":0.0067038030000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":8582,"num_write_kb":110246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'12334","reported_seq":12388,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.612993+0000","last_change":"2024-03-14T22:06:05.613124+0000","last_active":"2024-03-14T22:06:05.612993+0000","last_peered":"2024-03-14T22:06:05.612993+0000","last_clean":"2024-03-14T22:06:05.612993+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T22:06:05.612993+0000","last_undegraded":"2024-03-14T22:06:05.612993+0000","last_fullsized":"2024-03-14T22:06:05.612993+0000","mapping_epoch":19,"log_start":"23'2300","ondisk_log_start":"23'2300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12334","last_scrub_stamp":"2024-03-14T22:06:05.612968+0000","last_deep_scrub":"23'12334","last_deep_scrub_stamp":"2024-03-14T22:06:05.612968+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.612968+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:16.398011+0000","scrub_duration":0.010715311,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10697,"num_write_kb":121303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'9243","reported_seq":9299,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.500698+0000","last_change":"2024-03-14T22:06:01.500698+0000","last_active":"2024-03-14T22:06:01.500698+0000","last_peered":"2024-03-14T22:06:01.500698+0000","last_clean":"2024-03-14T22:06:01.500698+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T22:06:01.500698+0000","last_undegraded":"2024-03-14T22:06:01.500698+0000","last_fullsized":"2024-03-14T22:06:01.500698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9243","last_scrub_stamp":"2024-03-14T22:06:01.500545+0000","last_deep_scrub":"23'9243","last_deep_scrub_stamp":"2024-03-14T22:06:01.500545+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.500545+0000","objects_scrubbed":0,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:12.285588+0000","scrub_duration":0.0091242010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7601,"num_write_kb":100407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'10312","reported_seq":10360,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:49.938608+0000","last_change":"2024-03-14T22:05:49.938608+0000","last_active":"2024-03-14T22:05:49.938608+0000","last_peered":"2024-03-14T22:05:49.938608+0000","last_clean":"2024-03-14T22:05:49.938608+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T22:05:49.938608+0000","last_undegraded":"2024-03-14T22:05:49.938608+0000","last_fullsized":"2024-03-14T22:05:49.938608+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10312","last_scrub_stamp":"2024-03-14T22:05:49.938442+0000","last_deep_scrub":"23'10312","last_deep_scrub_stamp":"2024-03-14T22:05:49.938442+0000","last_clean_scrub_stamp":"2024-03-14T22:05:49.938442+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:20.664950+0000","scrub_duration":0.012238142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":108137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'11957","reported_seq":12017,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.174701+0000","last_change":"2024-03-14T22:06:01.174827+0000","last_active":"2024-03-14T22:06:01.174701+0000","last_peered":"2024-03-14T22:06:01.174701+0000","last_clean":"2024-03-14T22:06:01.174701+0000","last_became_active":"2024-03-14T21:14:44.746271+0000","last_became_peered":"2024-03-14T21:14:44.746271+0000","last_unstale":"2024-03-14T22:06:01.174701+0000","last_undegraded":"2024-03-14T22:06:01.174701+0000","last_fullsized":"2024-03-14T22:06:01.174701+0000","mapping_epoch":22,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11957","last_scrub_stamp":"2024-03-14T22:06:01.174676+0000","last_deep_scrub":"23'11957","last_deep_scrub_stamp":"2024-03-14T22:06:01.174676+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.174676+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:00.473961+0000","scrub_duration":0.0096213229999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":10306,"num_write_kb":120667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'11954","reported_seq":12009,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.635340+0000","last_change":"2024-03-14T22:06:05.635340+0000","last_active":"2024-03-14T22:06:05.635340+0000","last_peered":"2024-03-14T22:06:05.635340+0000","last_clean":"2024-03-14T22:06:05.635340+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T22:06:05.635340+0000","last_undegraded":"2024-03-14T22:06:05.635340+0000","last_fullsized":"2024-03-14T22:06:05.635340+0000","mapping_epoch":19,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11954","last_scrub_stamp":"2024-03-14T22:06:05.635217+0000","last_deep_scrub":"23'11954","last_deep_scrub_stamp":"2024-03-14T22:06:05.635217+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.635217+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:57.722318+0000","scrub_duration":0.012719486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10416,"num_write_kb":126678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'11296","reported_seq":11356,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.103982+0000","last_change":"2024-03-14T22:05:47.103982+0000","last_active":"2024-03-14T22:05:47.103982+0000","last_peered":"2024-03-14T22:05:47.103982+0000","last_clean":"2024-03-14T22:05:47.103982+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T22:05:47.103982+0000","last_undegraded":"2024-03-14T22:05:47.103982+0000","last_fullsized":"2024-03-14T22:05:47.103982+0000","mapping_epoch":19,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11296","last_scrub_stamp":"2024-03-14T22:05:47.103819+0000","last_deep_scrub":"23'11296","last_deep_scrub_stamp":"2024-03-14T22:05:47.103819+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.103819+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:17.830327+0000","scrub_duration":0.013053291999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2806,"num_write":9622,"num_write_kb":118224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'8637","reported_seq":8687,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:06.656145+0000","last_change":"2024-03-14T22:06:06.656145+0000","last_active":"2024-03-14T22:06:06.656145+0000","last_peered":"2024-03-14T22:06:06.656145+0000","last_clean":"2024-03-14T22:06:06.656145+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T22:06:06.656145+0000","last_undegraded":"2024-03-14T22:06:06.656145+0000","last_fullsized":"2024-03-14T22:06:06.656145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8637","last_scrub_stamp":"2024-03-14T22:06:06.655961+0000","last_deep_scrub":"23'8637","last_deep_scrub_stamp":"2024-03-14T22:06:06.655961+0000","last_clean_scrub_stamp":"2024-03-14T22:06:06.655961+0000","objects_scrubbed":0,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:00.034958+0000","scrub_duration":0.013905326000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":7051,"num_write_kb":105885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'9053","reported_seq":9101,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.818239+0000","last_change":"2024-03-14T22:05:37.818351+0000","last_active":"2024-03-14T22:05:37.818239+0000","last_peered":"2024-03-14T22:05:37.818239+0000","last_clean":"2024-03-14T22:05:37.818239+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T22:05:37.818239+0000","last_undegraded":"2024-03-14T22:05:37.818239+0000","last_fullsized":"2024-03-14T22:05:37.818239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9053","last_scrub_stamp":"2024-03-14T22:05:37.818216+0000","last_deep_scrub":"23'9053","last_deep_scrub_stamp":"2024-03-14T22:05:37.818216+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.818216+0000","objects_scrubbed":0,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:12:07.513721+0000","scrub_duration":0.014777571999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":7364,"num_write_kb":104642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'10865","reported_seq":10910,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.106755+0000","last_change":"2024-03-14T22:05:37.106859+0000","last_active":"2024-03-14T22:05:37.106755+0000","last_peered":"2024-03-14T22:05:37.106755+0000","last_clean":"2024-03-14T22:05:37.106755+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T22:05:37.106755+0000","last_undegraded":"2024-03-14T22:05:37.106755+0000","last_fullsized":"2024-03-14T22:05:37.106755+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10865","last_scrub_stamp":"2024-03-14T22:05:37.106731+0000","last_deep_scrub":"23'10865","last_deep_scrub_stamp":"2024-03-14T22:05:37.106731+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.106731+0000","objects_scrubbed":4,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:08.649923+0000","scrub_duration":0.054954579000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10855,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'9539","reported_seq":9590,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.102174+0000","last_change":"2024-03-14T22:05:43.102310+0000","last_active":"2024-03-14T22:05:43.102174+0000","last_peered":"2024-03-14T22:05:43.102174+0000","last_clean":"2024-03-14T22:05:43.102174+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T22:05:43.102174+0000","last_undegraded":"2024-03-14T22:05:43.102174+0000","last_fullsized":"2024-03-14T22:05:43.102174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9539","last_scrub_stamp":"2024-03-14T22:05:43.102152+0000","last_deep_scrub":"23'9539","last_deep_scrub_stamp":"2024-03-14T22:05:43.102152+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.102152+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:27.735439+0000","scrub_duration":0.010746225999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":7923,"num_write_kb":107722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10915","reported_seq":10960,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.455431+0000","last_change":"2024-03-14T22:05:43.455532+0000","last_active":"2024-03-14T22:05:43.455431+0000","last_peered":"2024-03-14T22:05:43.455431+0000","last_clean":"2024-03-14T22:05:43.455431+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T22:05:43.455431+0000","last_undegraded":"2024-03-14T22:05:43.455431+0000","last_fullsized":"2024-03-14T22:05:43.455431+0000","mapping_epoch":18,"log_start":"23'900","ondisk_log_start":"23'900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10915","last_scrub_stamp":"2024-03-14T22:05:43.455407+0000","last_deep_scrub":"23'10915","last_deep_scrub_stamp":"2024-03-14T22:05:43.455407+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.455407+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:28.088694+0000","scrub_duration":0.005473159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":10937,"num_write_kb":77181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'7037","reported_seq":7083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.042078+0000","last_change":"2024-03-14T22:05:38.042187+0000","last_active":"2024-03-14T22:05:38.042078+0000","last_peered":"2024-03-14T22:05:38.042078+0000","last_clean":"2024-03-14T22:05:38.042078+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T22:05:38.042078+0000","last_undegraded":"2024-03-14T22:05:38.042078+0000","last_fullsized":"2024-03-14T22:05:38.042078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7037","last_scrub_stamp":"2024-03-14T22:05:38.042054+0000","last_deep_scrub":"23'7037","last_deep_scrub_stamp":"2024-03-14T22:05:38.042054+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.042054+0000","objects_scrubbed":1,"log_size":7037,"log_dups_size":0,"ondisk_log_size":7037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:22.675341+0000","scrub_duration":0.019933317999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7046,"num_write_kb":58697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'11487","reported_seq":11533,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:00.969365+0000","last_change":"2024-03-14T22:06:00.969520+0000","last_active":"2024-03-14T22:06:00.969365+0000","last_peered":"2024-03-14T22:06:00.969365+0000","last_clean":"2024-03-14T22:06:00.969365+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T22:06:00.969365+0000","last_undegraded":"2024-03-14T22:06:00.969365+0000","last_fullsized":"2024-03-14T22:06:00.969365+0000","mapping_epoch":19,"log_start":"23'1400","ondisk_log_start":"23'1400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11487","last_scrub_stamp":"2024-03-14T22:06:00.969339+0000","last_deep_scrub":"23'11487","last_deep_scrub_stamp":"2024-03-14T22:06:00.969339+0000","last_clean_scrub_stamp":"2024-03-14T22:06:00.969339+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:54.348337+0000","scrub_duration":0.015493251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":9890,"num_write_kb":106254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'11077","reported_seq":11123,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:40.604311+0000","last_change":"2024-03-14T22:05:40.604410+0000","last_active":"2024-03-14T22:05:40.604311+0000","last_peered":"2024-03-14T22:05:40.604311+0000","last_clean":"2024-03-14T22:05:40.604311+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T22:05:40.604311+0000","last_undegraded":"2024-03-14T22:05:40.604311+0000","last_fullsized":"2024-03-14T22:05:40.604311+0000","mapping_epoch":18,"log_start":"23'1000","ondisk_log_start":"23'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11077","last_scrub_stamp":"2024-03-14T22:05:40.604286+0000","last_deep_scrub":"23'11077","last_deep_scrub_stamp":"2024-03-14T22:05:40.604286+0000","last_clean_scrub_stamp":"2024-03-14T22:05:40.604286+0000","objects_scrubbed":1,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:30.626988+0000","scrub_duration":0.021895357000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11078,"num_write_kb":81064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'10475","reported_seq":10531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.394404+0000","last_change":"2024-03-14T22:06:05.394533+0000","last_active":"2024-03-14T22:06:05.394404+0000","last_peered":"2024-03-14T22:06:05.394404+0000","last_clean":"2024-03-14T22:06:05.394404+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T22:06:05.394404+0000","last_undegraded":"2024-03-14T22:06:05.394404+0000","last_fullsized":"2024-03-14T22:06:05.394404+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10475","last_scrub_stamp":"2024-03-14T22:06:05.394380+0000","last_deep_scrub":"23'10475","last_deep_scrub_stamp":"2024-03-14T22:06:05.394380+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.394380+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:36.213410+0000","scrub_duration":0.0089458139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2800,"num_write":8909,"num_write_kb":114509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8287","reported_seq":8938,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:54.608435+0000","last_change":"2024-03-14T22:05:54.608544+0000","last_active":"2024-03-14T22:05:54.608435+0000","last_peered":"2024-03-14T22:05:54.608435+0000","last_clean":"2024-03-14T22:05:54.608435+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T22:05:54.608435+0000","last_undegraded":"2024-03-14T22:05:54.608435+0000","last_fullsized":"2024-03-14T22:05:54.608435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8287","last_scrub_stamp":"2024-03-14T22:05:54.608410+0000","last_deep_scrub":"23'8287","last_deep_scrub_stamp":"2024-03-14T22:05:54.608410+0000","last_clean_scrub_stamp":"2024-03-14T22:05:54.608410+0000","objects_scrubbed":1,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:36.637420+0000","scrub_duration":0.027372786,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":607,"num_read_kb":4494,"num_write":8292,"num_write_kb":66794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10203","reported_seq":10253,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:53.622084+0000","last_change":"2024-03-14T22:05:53.622208+0000","last_active":"2024-03-14T22:05:53.622084+0000","last_peered":"2024-03-14T22:05:53.622084+0000","last_clean":"2024-03-14T22:05:53.622084+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T22:05:53.622084+0000","last_undegraded":"2024-03-14T22:05:53.622084+0000","last_fullsized":"2024-03-14T22:05:53.622084+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10203","last_scrub_stamp":"2024-03-14T22:05:53.622058+0000","last_deep_scrub":"23'10203","last_deep_scrub_stamp":"2024-03-14T22:05:53.622058+0000","last_clean_scrub_stamp":"2024-03-14T22:05:53.622058+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:24.441088+0000","scrub_duration":0.015986779,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8609,"num_write_kb":104201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'10695","reported_seq":10741,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.657443+0000","last_change":"2024-03-14T22:05:39.657538+0000","last_active":"2024-03-14T22:05:39.657443+0000","last_peered":"2024-03-14T22:05:39.657443+0000","last_clean":"2024-03-14T22:05:39.657443+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T22:05:39.657443+0000","last_undegraded":"2024-03-14T22:05:39.657443+0000","last_fullsized":"2024-03-14T22:05:39.657443+0000","mapping_epoch":18,"log_start":"23'600","ondisk_log_start":"23'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10695","last_scrub_stamp":"2024-03-14T22:05:39.657419+0000","last_deep_scrub":"23'10695","last_deep_scrub_stamp":"2024-03-14T22:05:39.657419+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.657419+0000","objects_scrubbed":6,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:10.383927+0000","scrub_duration":0.080830166999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":92935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'9099","reported_seq":9148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:53.162311+0000","last_change":"2024-03-14T22:05:53.162394+0000","last_active":"2024-03-14T22:05:53.162311+0000","last_peered":"2024-03-14T22:05:53.162311+0000","last_clean":"2024-03-14T22:05:53.162311+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T22:05:53.162311+0000","last_undegraded":"2024-03-14T22:05:53.162311+0000","last_fullsized":"2024-03-14T22:05:53.162311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9099","last_scrub_stamp":"2024-03-14T22:05:53.162292+0000","last_deep_scrub":"23'9099","last_deep_scrub_stamp":"2024-03-14T22:05:53.162292+0000","last_clean_scrub_stamp":"2024-03-14T22:05:53.162292+0000","objects_scrubbed":0,"log_size":9099,"log_dups_size":0,"ondisk_log_size":9099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:35.191303+0000","scrub_duration":0.0086207539999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7455,"num_write_kb":104328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9150","reported_seq":9196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:51.453255+0000","last_change":"2024-03-14T22:05:51.453370+0000","last_active":"2024-03-14T22:05:51.453255+0000","last_peered":"2024-03-14T22:05:51.453255+0000","last_clean":"2024-03-14T22:05:51.453255+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T22:05:51.453255+0000","last_undegraded":"2024-03-14T22:05:51.453255+0000","last_fullsized":"2024-03-14T22:05:51.453255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9150","last_scrub_stamp":"2024-03-14T22:05:51.453231+0000","last_deep_scrub":"23'9150","last_deep_scrub_stamp":"2024-03-14T22:05:51.453231+0000","last_clean_scrub_stamp":"2024-03-14T22:05:51.453231+0000","objects_scrubbed":2,"log_size":9150,"log_dups_size":0,"ondisk_log_size":9150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:22.179739+0000","scrub_duration":0.01173714,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9159,"num_write_kb":60302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11329","reported_seq":11385,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.792145+0000","last_change":"2024-03-14T22:05:34.792145+0000","last_active":"2024-03-14T22:05:34.792145+0000","last_peered":"2024-03-14T22:05:34.792145+0000","last_clean":"2024-03-14T22:05:34.792145+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T22:05:34.792145+0000","last_undegraded":"2024-03-14T22:05:34.792145+0000","last_fullsized":"2024-03-14T22:05:34.792145+0000","mapping_epoch":19,"log_start":"23'1300","ondisk_log_start":"23'1300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11329","last_scrub_stamp":"2024-03-14T22:05:34.791977+0000","last_deep_scrub":"23'11329","last_deep_scrub_stamp":"2024-03-14T22:05:34.791977+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.791977+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:36:18.331402+0000","scrub_duration":0.011767835000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2542,"num_write":9699,"num_write_kb":113170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'13354","reported_seq":13955,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.451109+0000","last_change":"2024-03-14T22:05:38.451212+0000","last_active":"2024-03-14T22:05:38.451109+0000","last_peered":"2024-03-14T22:05:38.451109+0000","last_clean":"2024-03-14T22:05:38.451109+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T22:05:38.451109+0000","last_undegraded":"2024-03-14T22:05:38.451109+0000","last_fullsized":"2024-03-14T22:05:38.451109+0000","mapping_epoch":18,"log_start":"23'3300","ondisk_log_start":"23'3300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13354","last_scrub_stamp":"2024-03-14T22:05:38.451087+0000","last_deep_scrub":"23'13354","last_deep_scrub_stamp":"2024-03-14T22:05:38.451087+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.451087+0000","objects_scrubbed":4,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:50.713752+0000","scrub_duration":0.026780381999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":557,"num_read_kb":4455,"num_write":13349,"num_write_kb":107014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'9106","reported_seq":9153,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.805465+0000","last_change":"2024-03-14T21:14:03.032177+0000","last_active":"2024-03-14T22:05:25.805465+0000","last_peered":"2024-03-14T22:05:25.805465+0000","last_clean":"2024-03-14T22:05:25.805465+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T22:05:25.805465+0000","last_undegraded":"2024-03-14T22:05:25.805465+0000","last_fullsized":"2024-03-14T22:05:25.805465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9106,"log_dups_size":0,"ondisk_log_size":9106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3318,"num_write":7536,"num_write_kb":95248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'8011","reported_seq":8058,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.179662+0000","last_change":"2024-03-14T22:05:37.179778+0000","last_active":"2024-03-14T22:05:37.179662+0000","last_peered":"2024-03-14T22:05:37.179662+0000","last_clean":"2024-03-14T22:05:37.179662+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T22:05:37.179662+0000","last_undegraded":"2024-03-14T22:05:37.179662+0000","last_fullsized":"2024-03-14T22:05:37.179662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8011","last_scrub_stamp":"2024-03-14T22:05:37.179641+0000","last_deep_scrub":"23'8011","last_deep_scrub_stamp":"2024-03-14T22:05:37.179641+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.179641+0000","objects_scrubbed":2,"log_size":8011,"log_dups_size":0,"ondisk_log_size":8011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:49.442306+0000","scrub_duration":0.011524995999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8008,"num_write_kb":62700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'9649","reported_seq":9700,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:54.202635+0000","last_change":"2024-03-14T22:05:54.202762+0000","last_active":"2024-03-14T22:05:54.202635+0000","last_peered":"2024-03-14T22:05:54.202635+0000","last_clean":"2024-03-14T22:05:54.202635+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T22:05:54.202635+0000","last_undegraded":"2024-03-14T22:05:54.202635+0000","last_fullsized":"2024-03-14T22:05:54.202635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9649","last_scrub_stamp":"2024-03-14T22:05:54.202612+0000","last_deep_scrub":"23'9649","last_deep_scrub_stamp":"2024-03-14T22:05:54.202612+0000","last_clean_scrub_stamp":"2024-03-14T22:05:54.202612+0000","objects_scrubbed":0,"log_size":9649,"log_dups_size":0,"ondisk_log_size":9649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:24.929120+0000","scrub_duration":0.013083743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8033,"num_write_kb":100651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5758","reported_seq":5805,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:45.712184+0000","last_change":"2024-03-14T22:05:45.712286+0000","last_active":"2024-03-14T22:05:45.712184+0000","last_peered":"2024-03-14T22:05:45.712184+0000","last_clean":"2024-03-14T22:05:45.712184+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T22:05:45.712184+0000","last_undegraded":"2024-03-14T22:05:45.712184+0000","last_fullsized":"2024-03-14T22:05:45.712184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5758","last_scrub_stamp":"2024-03-14T22:05:45.712161+0000","last_deep_scrub":"23'5758","last_deep_scrub_stamp":"2024-03-14T22:05:45.712161+0000","last_clean_scrub_stamp":"2024-03-14T22:05:45.712161+0000","objects_scrubbed":2,"log_size":5758,"log_dups_size":0,"ondisk_log_size":5758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:45.011446+0000","scrub_duration":0.027383683999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5754,"num_write_kb":48600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'9764","reported_seq":9812,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:02.491868+0000","last_change":"2024-03-14T22:06:02.492011+0000","last_active":"2024-03-14T22:06:02.491868+0000","last_peered":"2024-03-14T22:06:02.491868+0000","last_clean":"2024-03-14T22:06:02.491868+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T22:06:02.491868+0000","last_undegraded":"2024-03-14T22:06:02.491868+0000","last_fullsized":"2024-03-14T22:06:02.491868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9764","last_scrub_stamp":"2024-03-14T22:06:02.491844+0000","last_deep_scrub":"23'9764","last_deep_scrub_stamp":"2024-03-14T22:06:02.491844+0000","last_clean_scrub_stamp":"2024-03-14T22:06:02.491844+0000","objects_scrubbed":0,"log_size":9764,"log_dups_size":0,"ondisk_log_size":9764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:27:54.242135+0000","scrub_duration":0.011674722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8135,"num_write_kb":106503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5852","reported_seq":5899,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.442569+0000","last_change":"2024-03-14T22:05:42.442683+0000","last_active":"2024-03-14T22:05:42.442569+0000","last_peered":"2024-03-14T22:05:42.442569+0000","last_clean":"2024-03-14T22:05:42.442569+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T22:05:42.442569+0000","last_undegraded":"2024-03-14T22:05:42.442569+0000","last_fullsized":"2024-03-14T22:05:42.442569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5852","last_scrub_stamp":"2024-03-14T22:05:42.442545+0000","last_deep_scrub":"23'5852","last_deep_scrub_stamp":"2024-03-14T22:05:42.442545+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.442545+0000","objects_scrubbed":1,"log_size":5852,"log_dups_size":0,"ondisk_log_size":5852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:13.985738+0000","scrub_duration":0.016901003000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":52920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'9086","reported_seq":9142,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:55.633873+0000","last_change":"2024-03-14T22:05:55.634080+0000","last_active":"2024-03-14T22:05:55.633873+0000","last_peered":"2024-03-14T22:05:55.633873+0000","last_clean":"2024-03-14T22:05:55.633873+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T22:05:55.633873+0000","last_undegraded":"2024-03-14T22:05:55.633873+0000","last_fullsized":"2024-03-14T22:05:55.633873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9086","last_scrub_stamp":"2024-03-14T22:05:55.633836+0000","last_deep_scrub":"23'9086","last_deep_scrub_stamp":"2024-03-14T22:05:55.633836+0000","last_clean_scrub_stamp":"2024-03-14T22:05:55.633836+0000","objects_scrubbed":0,"log_size":9086,"log_dups_size":0,"ondisk_log_size":9086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:26.360344+0000","scrub_duration":0.014642007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2548,"num_write":7495,"num_write_kb":101746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8334","reported_seq":8381,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:30.179949+0000","last_change":"2024-03-14T22:05:30.180079+0000","last_active":"2024-03-14T22:05:30.179949+0000","last_peered":"2024-03-14T22:05:30.179949+0000","last_clean":"2024-03-14T22:05:30.179949+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T22:05:30.179949+0000","last_undegraded":"2024-03-14T22:05:30.179949+0000","last_fullsized":"2024-03-14T22:05:30.179949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8334","last_scrub_stamp":"2024-03-14T22:05:30.179928+0000","last_deep_scrub":"23'8334","last_deep_scrub_stamp":"2024-03-14T22:05:30.179928+0000","last_clean_scrub_stamp":"2024-03-14T22:05:30.179928+0000","objects_scrubbed":8,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:42.442593+0000","scrub_duration":0.057801277999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16724139,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":8340,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'10136","reported_seq":10184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:06.386397+0000","last_change":"2024-03-14T22:06:06.386528+0000","last_active":"2024-03-14T22:06:06.386397+0000","last_peered":"2024-03-14T22:06:06.386397+0000","last_clean":"2024-03-14T22:06:06.386397+0000","last_became_active":"2024-03-14T21:14:44.859185+0000","last_became_peered":"2024-03-14T21:14:44.859185+0000","last_unstale":"2024-03-14T22:06:06.386397+0000","last_undegraded":"2024-03-14T22:06:06.386397+0000","last_fullsized":"2024-03-14T22:06:06.386397+0000","mapping_epoch":22,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10136","last_scrub_stamp":"2024-03-14T22:06:06.386374+0000","last_deep_scrub":"23'10136","last_deep_scrub_stamp":"2024-03-14T22:06:06.386374+0000","last_clean_scrub_stamp":"2024-03-14T22:06:06.386374+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:23:26.419647+0000","scrub_duration":0.013047168,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":8491,"num_write_kb":113136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"23'8049","reported_seq":8097,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.782835+0000","last_change":"2024-03-14T22:05:31.782835+0000","last_active":"2024-03-14T22:05:31.782835+0000","last_peered":"2024-03-14T22:05:31.782835+0000","last_clean":"2024-03-14T22:05:31.782835+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T22:05:31.782835+0000","last_undegraded":"2024-03-14T22:05:31.782835+0000","last_fullsized":"2024-03-14T22:05:31.782835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8049","last_scrub_stamp":"2024-03-14T22:05:31.782651+0000","last_deep_scrub":"23'8049","last_deep_scrub_stamp":"2024-03-14T22:05:31.782651+0000","last_clean_scrub_stamp":"2024-03-14T22:05:31.782651+0000","objects_scrubbed":3,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:57:34.557091+0000","scrub_duration":0.036800939999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8051,"num_write_kb":62569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'9555","reported_seq":9610,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.307278+0000","last_change":"2024-03-14T22:05:34.307278+0000","last_active":"2024-03-14T22:05:34.307278+0000","last_peered":"2024-03-14T22:05:34.307278+0000","last_clean":"2024-03-14T22:05:34.307278+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T22:05:34.307278+0000","last_undegraded":"2024-03-14T22:05:34.307278+0000","last_fullsized":"2024-03-14T22:05:34.307278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9555","last_scrub_stamp":"2024-03-14T22:05:34.307098+0000","last_deep_scrub":"23'9555","last_deep_scrub_stamp":"2024-03-14T22:05:34.307098+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.307098+0000","objects_scrubbed":0,"log_size":9555,"log_dups_size":0,"ondisk_log_size":9555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:18.940385+0000","scrub_duration":0.012363498000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2042,"num_write":7926,"num_write_kb":109666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'10891","reported_seq":10939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.205897+0000","last_change":"2024-03-14T22:05:39.205897+0000","last_active":"2024-03-14T22:05:39.205897+0000","last_peered":"2024-03-14T22:05:39.205897+0000","last_clean":"2024-03-14T22:05:39.205897+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T22:05:39.205897+0000","last_undegraded":"2024-03-14T22:05:39.205897+0000","last_fullsized":"2024-03-14T22:05:39.205897+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10891","last_scrub_stamp":"2024-03-14T22:05:39.205752+0000","last_deep_scrub":"23'10891","last_deep_scrub_stamp":"2024-03-14T22:05:39.205752+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.205752+0000","objects_scrubbed":2,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:13.412709+0000","scrub_duration":0.023607316999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10896,"num_write_kb":88926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9015","reported_seq":9056,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:32.411619+0000","last_change":"2024-03-14T21:14:03.037858+0000","last_active":"2024-03-14T22:05:32.411619+0000","last_peered":"2024-03-14T22:05:32.411619+0000","last_clean":"2024-03-14T22:05:32.411619+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T22:05:32.411619+0000","last_undegraded":"2024-03-14T22:05:32.411619+0000","last_fullsized":"2024-03-14T22:05:32.411619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":9015,"log_dups_size":0,"ondisk_log_size":9015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":7443,"num_write_kb":93234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'11502","reported_seq":11550,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.317932+0000","last_change":"2024-03-14T22:05:39.317932+0000","last_active":"2024-03-14T22:05:39.317932+0000","last_peered":"2024-03-14T22:05:39.317932+0000","last_clean":"2024-03-14T22:05:39.317932+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T22:05:39.317932+0000","last_undegraded":"2024-03-14T22:05:39.317932+0000","last_fullsized":"2024-03-14T22:05:39.317932+0000","mapping_epoch":18,"log_start":"23'1500","ondisk_log_start":"23'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11502","last_scrub_stamp":"2024-03-14T22:05:39.317814+0000","last_deep_scrub":"23'11502","last_deep_scrub_stamp":"2024-03-14T22:05:39.317814+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.317814+0000","objects_scrubbed":4,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:23.951100+0000","scrub_duration":0.049637907000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11487,"num_write_kb":85212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'9516","reported_seq":9569,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:52.659321+0000","last_change":"2024-03-14T22:05:52.659529+0000","last_active":"2024-03-14T22:05:52.659321+0000","last_peered":"2024-03-14T22:05:52.659321+0000","last_clean":"2024-03-14T22:05:52.659321+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T22:05:52.659321+0000","last_undegraded":"2024-03-14T22:05:52.659321+0000","last_fullsized":"2024-03-14T22:05:52.659321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9516","last_scrub_stamp":"2024-03-14T22:05:52.659298+0000","last_deep_scrub":"23'9516","last_deep_scrub_stamp":"2024-03-14T22:05:52.659298+0000","last_clean_scrub_stamp":"2024-03-14T22:05:52.659298+0000","objects_scrubbed":0,"log_size":9516,"log_dups_size":0,"ondisk_log_size":9516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:11.394549+0000","scrub_duration":0.012300805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":7940,"num_write_kb":115635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8906","reported_seq":8954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.143252+0000","last_change":"2024-03-14T22:05:34.143252+0000","last_active":"2024-03-14T22:05:34.143252+0000","last_peered":"2024-03-14T22:05:34.143252+0000","last_clean":"2024-03-14T22:05:34.143252+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T22:05:34.143252+0000","last_undegraded":"2024-03-14T22:05:34.143252+0000","last_fullsized":"2024-03-14T22:05:34.143252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8906","last_scrub_stamp":"2024-03-14T22:05:34.143115+0000","last_deep_scrub":"23'8906","last_deep_scrub_stamp":"2024-03-14T22:05:34.143115+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.143115+0000","objects_scrubbed":2,"log_size":8906,"log_dups_size":0,"ondisk_log_size":8906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:05.686307+0000","scrub_duration":0.021707378999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8892,"num_write_kb":67382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1170,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'9427","reported_seq":9478,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:09.484008+0000","last_change":"2024-03-14T22:06:09.484137+0000","last_active":"2024-03-14T22:06:09.484008+0000","last_peered":"2024-03-14T22:06:09.484008+0000","last_clean":"2024-03-14T22:06:09.484008+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T22:06:09.484008+0000","last_undegraded":"2024-03-14T22:06:09.484008+0000","last_fullsized":"2024-03-14T22:06:09.484008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9427","last_scrub_stamp":"2024-03-14T22:06:09.483985+0000","last_deep_scrub":"23'9427","last_deep_scrub_stamp":"2024-03-14T22:06:09.483985+0000","last_clean_scrub_stamp":"2024-03-14T22:06:09.483985+0000","objects_scrubbed":0,"log_size":9427,"log_dups_size":0,"ondisk_log_size":9427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:19:27.695841+0000","scrub_duration":0.011868826000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7820,"num_write_kb":110543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.280716+0000","last_change":"2024-03-14T22:05:38.280716+0000","last_active":"2024-03-14T22:05:38.280716+0000","last_peered":"2024-03-14T22:05:38.280716+0000","last_clean":"2024-03-14T22:05:38.280716+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T22:05:38.280716+0000","last_undegraded":"2024-03-14T22:05:38.280716+0000","last_fullsized":"2024-03-14T22:05:38.280716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-03-14T22:05:38.280592+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-03-14T22:05:38.280592+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.280592+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:12.487548+0000","scrub_duration":0.0062042419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'8579","reported_seq":8629,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:32.525126+0000","last_change":"2024-03-14T22:05:32.525126+0000","last_active":"2024-03-14T22:05:32.525126+0000","last_peered":"2024-03-14T22:05:32.525126+0000","last_clean":"2024-03-14T22:05:32.525126+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T22:05:32.525126+0000","last_undegraded":"2024-03-14T22:05:32.525126+0000","last_fullsized":"2024-03-14T22:05:32.525126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8579","last_scrub_stamp":"2024-03-14T22:05:32.525005+0000","last_deep_scrub":"23'8579","last_deep_scrub_stamp":"2024-03-14T22:05:32.525005+0000","last_clean_scrub_stamp":"2024-03-14T22:05:32.525005+0000","objects_scrubbed":2,"log_size":8579,"log_dups_size":0,"ondisk_log_size":8579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:06.731961+0000","scrub_duration":0.031238887999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":67724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'11133","reported_seq":11185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:04.961103+0000","last_change":"2024-03-14T22:06:04.961103+0000","last_active":"2024-03-14T22:06:04.961103+0000","last_peered":"2024-03-14T22:06:04.961103+0000","last_clean":"2024-03-14T22:06:04.961103+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T22:06:04.961103+0000","last_undegraded":"2024-03-14T22:06:04.961103+0000","last_fullsized":"2024-03-14T22:06:04.961103+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11133","last_scrub_stamp":"2024-03-14T22:06:04.960946+0000","last_deep_scrub":"23'11133","last_deep_scrub_stamp":"2024-03-14T22:06:04.960946+0000","last_clean_scrub_stamp":"2024-03-14T22:06:04.960946+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:23:24.994219+0000","scrub_duration":0.013835398,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":9507,"num_write_kb":117178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'8677","reported_seq":8727,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:52.522888+0000","last_change":"2024-03-14T22:05:52.523012+0000","last_active":"2024-03-14T22:05:52.522888+0000","last_peered":"2024-03-14T22:05:52.522888+0000","last_clean":"2024-03-14T22:05:52.522888+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T22:05:52.522888+0000","last_undegraded":"2024-03-14T22:05:52.522888+0000","last_fullsized":"2024-03-14T22:05:52.522888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8677","last_scrub_stamp":"2024-03-14T22:05:52.522865+0000","last_deep_scrub":"23'8677","last_deep_scrub_stamp":"2024-03-14T22:05:52.522865+0000","last_clean_scrub_stamp":"2024-03-14T22:05:52.522865+0000","objects_scrubbed":1,"log_size":8677,"log_dups_size":0,"ondisk_log_size":8677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:51.822151+0000","scrub_duration":0.0171749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8672,"num_write_kb":67194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10246","reported_seq":10297,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.131437+0000","last_change":"2024-03-14T22:05:42.131437+0000","last_active":"2024-03-14T22:05:42.131437+0000","last_peered":"2024-03-14T22:05:42.131437+0000","last_clean":"2024-03-14T22:05:42.131437+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T22:05:42.131437+0000","last_undegraded":"2024-03-14T22:05:42.131437+0000","last_fullsized":"2024-03-14T22:05:42.131437+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10246","last_scrub_stamp":"2024-03-14T22:05:42.131282+0000","last_deep_scrub":"23'10246","last_deep_scrub_stamp":"2024-03-14T22:05:42.131282+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.131282+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:13.674474+0000","scrub_duration":0.011884756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8656,"num_write_kb":115905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11306","reported_seq":12023,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:35.173769+0000","last_change":"2024-03-14T22:05:35.173769+0000","last_active":"2024-03-14T22:05:35.173769+0000","last_peered":"2024-03-14T22:05:35.173769+0000","last_clean":"2024-03-14T22:05:35.173769+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T22:05:35.173769+0000","last_undegraded":"2024-03-14T22:05:35.173769+0000","last_fullsized":"2024-03-14T22:05:35.173769+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11306","last_scrub_stamp":"2024-03-14T22:05:35.173652+0000","last_deep_scrub":"23'11306","last_deep_scrub_stamp":"2024-03-14T22:05:35.173652+0000","last_clean_scrub_stamp":"2024-03-14T22:05:35.173652+0000","objects_scrubbed":3,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:19.806939+0000","scrub_duration":0.046066214000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":669,"num_read_kb":4516,"num_write":11296,"num_write_kb":89038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'9443","reported_seq":9496,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:02.462769+0000","last_change":"2024-03-14T22:06:02.462769+0000","last_active":"2024-03-14T22:06:02.462769+0000","last_peered":"2024-03-14T22:06:02.462769+0000","last_clean":"2024-03-14T22:06:02.462769+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T22:06:02.462769+0000","last_undegraded":"2024-03-14T22:06:02.462769+0000","last_fullsized":"2024-03-14T22:06:02.462769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9443","last_scrub_stamp":"2024-03-14T22:06:02.462621+0000","last_deep_scrub":"23'9443","last_deep_scrub_stamp":"2024-03-14T22:06:02.462621+0000","last_clean_scrub_stamp":"2024-03-14T22:06:02.462621+0000","objects_scrubbed":0,"log_size":9443,"log_dups_size":0,"ondisk_log_size":9443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:55.841618+0000","scrub_duration":0.011137756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7837,"num_write_kb":123513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'9296","reported_seq":9346,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.595654+0000","last_change":"2024-03-14T22:05:38.595654+0000","last_active":"2024-03-14T22:05:38.595654+0000","last_peered":"2024-03-14T22:05:38.595654+0000","last_clean":"2024-03-14T22:05:38.595654+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T22:05:38.595654+0000","last_undegraded":"2024-03-14T22:05:38.595654+0000","last_fullsized":"2024-03-14T22:05:38.595654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9296","last_scrub_stamp":"2024-03-14T22:05:38.595537+0000","last_deep_scrub":"23'9296","last_deep_scrub_stamp":"2024-03-14T22:05:38.595537+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.595537+0000","objects_scrubbed":0,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:20.624547+0000","scrub_duration":0.0060746669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9294,"num_write_kb":71830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'9546","reported_seq":9596,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.855707+0000","last_change":"2024-03-14T22:05:43.855707+0000","last_active":"2024-03-14T22:05:43.855707+0000","last_peered":"2024-03-14T22:05:43.855707+0000","last_clean":"2024-03-14T22:05:43.855707+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T22:05:43.855707+0000","last_undegraded":"2024-03-14T22:05:43.855707+0000","last_fullsized":"2024-03-14T22:05:43.855707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9546","last_scrub_stamp":"2024-03-14T22:05:43.855569+0000","last_deep_scrub":"23'9546","last_deep_scrub_stamp":"2024-03-14T22:05:43.855569+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.855569+0000","objects_scrubbed":0,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:35:00.547431+0000","scrub_duration":0.01173973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":7936,"num_write_kb":109238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'11136","reported_seq":11186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:50.431205+0000","last_change":"2024-03-14T22:05:50.431205+0000","last_active":"2024-03-14T22:05:50.431205+0000","last_peered":"2024-03-14T22:05:50.431205+0000","last_clean":"2024-03-14T22:05:50.431205+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T22:05:50.431205+0000","last_undegraded":"2024-03-14T22:05:50.431205+0000","last_fullsized":"2024-03-14T22:05:50.431205+0000","mapping_epoch":18,"log_start":"23'1100","ondisk_log_start":"23'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11136","last_scrub_stamp":"2024-03-14T22:05:50.431052+0000","last_deep_scrub":"23'11136","last_deep_scrub_stamp":"2024-03-14T22:05:50.431052+0000","last_clean_scrub_stamp":"2024-03-14T22:05:50.431052+0000","objects_scrubbed":2,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:32.460062+0000","scrub_duration":0.031083458000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11130,"num_write_kb":85841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'12171","reported_seq":12219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:50.718433+0000","last_change":"2024-03-14T22:05:50.718554+0000","last_active":"2024-03-14T22:05:50.718433+0000","last_peered":"2024-03-14T22:05:50.718433+0000","last_clean":"2024-03-14T22:05:50.718433+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T22:05:50.718433+0000","last_undegraded":"2024-03-14T22:05:50.718433+0000","last_fullsized":"2024-03-14T22:05:50.718433+0000","mapping_epoch":19,"log_start":"23'2100","ondisk_log_start":"23'2100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12171","last_scrub_stamp":"2024-03-14T22:05:50.718411+0000","last_deep_scrub":"23'12171","last_deep_scrub_stamp":"2024-03-14T22:05:50.718411+0000","last_clean_scrub_stamp":"2024-03-14T22:05:50.718411+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:44.097408+0000","scrub_duration":0.014855348000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10560,"num_write_kb":124816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'9284","reported_seq":9333,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.431802+0000","last_change":"2024-03-14T22:05:47.431939+0000","last_active":"2024-03-14T22:05:47.431802+0000","last_peered":"2024-03-14T22:05:47.431802+0000","last_clean":"2024-03-14T22:05:47.431802+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T22:05:47.431802+0000","last_undegraded":"2024-03-14T22:05:47.431802+0000","last_fullsized":"2024-03-14T22:05:47.431802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9284","last_scrub_stamp":"2024-03-14T22:05:47.431780+0000","last_deep_scrub":"23'9284","last_deep_scrub_stamp":"2024-03-14T22:05:47.431780+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.431780+0000","objects_scrubbed":1,"log_size":9284,"log_dups_size":0,"ondisk_log_size":9284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:37.454481+0000","scrub_duration":0.019347982999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9294,"num_write_kb":71728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9772","reported_seq":9826,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:57.488903+0000","last_change":"2024-03-14T22:05:57.488903+0000","last_active":"2024-03-14T22:05:57.488903+0000","last_peered":"2024-03-14T22:05:57.488903+0000","last_clean":"2024-03-14T22:05:57.488903+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T22:05:57.488903+0000","last_undegraded":"2024-03-14T22:05:57.488903+0000","last_fullsized":"2024-03-14T22:05:57.488903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9772","last_scrub_stamp":"2024-03-14T22:05:57.488748+0000","last_deep_scrub":"23'9772","last_deep_scrub_stamp":"2024-03-14T22:05:57.488748+0000","last_clean_scrub_stamp":"2024-03-14T22:05:57.488748+0000","objects_scrubbed":0,"log_size":9772,"log_dups_size":0,"ondisk_log_size":9772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:56.788033+0000","scrub_duration":0.013426983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8091,"num_write_kb":107879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'8360","reported_seq":9002,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.381548+0000","last_change":"2024-03-14T22:05:38.381548+0000","last_active":"2024-03-14T22:05:38.381548+0000","last_peered":"2024-03-14T22:05:38.381548+0000","last_clean":"2024-03-14T22:05:38.381548+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T22:05:38.381548+0000","last_undegraded":"2024-03-14T22:05:38.381548+0000","last_fullsized":"2024-03-14T22:05:38.381548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8360","last_scrub_stamp":"2024-03-14T22:05:38.381407+0000","last_deep_scrub":"23'8360","last_deep_scrub_stamp":"2024-03-14T22:05:38.381407+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.381407+0000","objects_scrubbed":2,"log_size":8360,"log_dups_size":0,"ondisk_log_size":8360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:09.924599+0000","scrub_duration":0.064290047000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":593,"num_read_kb":4491,"num_write":9534,"num_write_kb":73072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'10081","reported_seq":10138,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.020373+0000","last_change":"2024-03-14T22:05:42.020373+0000","last_active":"2024-03-14T22:05:42.020373+0000","last_peered":"2024-03-14T22:05:42.020373+0000","last_clean":"2024-03-14T22:05:42.020373+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T22:05:42.020373+0000","last_undegraded":"2024-03-14T22:05:42.020373+0000","last_fullsized":"2024-03-14T22:05:42.020373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10081","last_scrub_stamp":"2024-03-14T22:05:42.020207+0000","last_deep_scrub":"23'10081","last_deep_scrub_stamp":"2024-03-14T22:05:42.020207+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.020207+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:16.227164+0000","scrub_duration":0.010075914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8532,"num_write_kb":114847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5882","reported_seq":5931,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.478496+0000","last_change":"2024-03-14T22:05:47.478496+0000","last_active":"2024-03-14T22:05:47.478496+0000","last_peered":"2024-03-14T22:05:47.478496+0000","last_clean":"2024-03-14T22:05:47.478496+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T22:05:47.478496+0000","last_undegraded":"2024-03-14T22:05:47.478496+0000","last_fullsized":"2024-03-14T22:05:47.478496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5882","last_scrub_stamp":"2024-03-14T22:05:47.478380+0000","last_deep_scrub":"23'5882","last_deep_scrub_stamp":"2024-03-14T22:05:47.478380+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.478380+0000","objects_scrubbed":2,"log_size":5882,"log_dups_size":0,"ondisk_log_size":5882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:21.685337+0000","scrub_duration":0.023401211000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5885,"num_write_kb":44672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17052,"num_omap_keys":21,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'10915","reported_seq":10970,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.120809+0000","last_change":"2024-03-14T22:05:47.120809+0000","last_active":"2024-03-14T22:05:47.120809+0000","last_peered":"2024-03-14T22:05:47.120809+0000","last_clean":"2024-03-14T22:05:47.120809+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T22:05:47.120809+0000","last_undegraded":"2024-03-14T22:05:47.120809+0000","last_fullsized":"2024-03-14T22:05:47.120809+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10915","last_scrub_stamp":"2024-03-14T22:05:47.120658+0000","last_deep_scrub":"23'10915","last_deep_scrub_stamp":"2024-03-14T22:05:47.120658+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.120658+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:21.327615+0000","scrub_duration":0.011394476000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":9274,"num_write_kb":112928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'4363","reported_seq":4412,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:56.577952+0000","last_change":"2024-03-14T22:05:56.577952+0000","last_active":"2024-03-14T22:05:56.577952+0000","last_peered":"2024-03-14T22:05:56.577952+0000","last_clean":"2024-03-14T22:05:56.577952+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T22:05:56.577952+0000","last_undegraded":"2024-03-14T22:05:56.577952+0000","last_fullsized":"2024-03-14T22:05:56.577952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4363","last_scrub_stamp":"2024-03-14T22:05:56.577822+0000","last_deep_scrub":"23'4363","last_deep_scrub_stamp":"2024-03-14T22:05:56.577822+0000","last_clean_scrub_stamp":"2024-03-14T22:05:56.577822+0000","objects_scrubbed":3,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:07.362865+0000","scrub_duration":0.035695787999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4362,"num_write_kb":43693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'9887","reported_seq":9948,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:40.803829+0000","last_change":"2024-03-14T22:05:40.803829+0000","last_active":"2024-03-14T22:05:40.803829+0000","last_peered":"2024-03-14T22:05:40.803829+0000","last_clean":"2024-03-14T22:05:40.803829+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T22:05:40.803829+0000","last_undegraded":"2024-03-14T22:05:40.803829+0000","last_fullsized":"2024-03-14T22:05:40.803829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9887","last_scrub_stamp":"2024-03-14T22:05:40.803679+0000","last_deep_scrub":"23'9887","last_deep_scrub_stamp":"2024-03-14T22:05:40.803679+0000","last_clean_scrub_stamp":"2024-03-14T22:05:40.803679+0000","objects_scrubbed":0,"log_size":9887,"log_dups_size":0,"ondisk_log_size":9887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:46:25.442439+0000","scrub_duration":0.011556728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":8266,"num_write_kb":116331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'6408","reported_seq":6457,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.393658+0000","last_change":"2024-03-14T22:05:43.393658+0000","last_active":"2024-03-14T22:05:43.393658+0000","last_peered":"2024-03-14T22:05:43.393658+0000","last_clean":"2024-03-14T22:05:43.393658+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T22:05:43.393658+0000","last_undegraded":"2024-03-14T22:05:43.393658+0000","last_fullsized":"2024-03-14T22:05:43.393658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6408","last_scrub_stamp":"2024-03-14T22:05:43.393505+0000","last_deep_scrub":"23'6408","last_deep_scrub_stamp":"2024-03-14T22:05:43.393505+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.393505+0000","objects_scrubbed":2,"log_size":6408,"log_dups_size":0,"ondisk_log_size":6408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:17.600461+0000","scrub_duration":0.029677097,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6417,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'10927","reported_seq":10975,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:48.972164+0000","last_change":"2024-03-14T22:05:48.972164+0000","last_active":"2024-03-14T22:05:48.972164+0000","last_peered":"2024-03-14T22:05:48.972164+0000","last_clean":"2024-03-14T22:05:48.972164+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T22:05:48.972164+0000","last_undegraded":"2024-03-14T22:05:48.972164+0000","last_fullsized":"2024-03-14T22:05:48.972164+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10927","last_scrub_stamp":"2024-03-14T22:05:48.972019+0000","last_deep_scrub":"23'10927","last_deep_scrub_stamp":"2024-03-14T22:05:48.972019+0000","last_clean_scrub_stamp":"2024-03-14T22:05:48.972019+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:31.001029+0000","scrub_duration":0.013693627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":120976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'9353","reported_seq":9402,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.295195+0000","last_change":"2024-03-14T22:05:34.295195+0000","last_active":"2024-03-14T22:05:34.295195+0000","last_peered":"2024-03-14T22:05:34.295195+0000","last_clean":"2024-03-14T22:05:34.295195+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T22:05:34.295195+0000","last_undegraded":"2024-03-14T22:05:34.295195+0000","last_fullsized":"2024-03-14T22:05:34.295195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9353","last_scrub_stamp":"2024-03-14T22:05:34.295063+0000","last_deep_scrub":"23'9353","last_deep_scrub_stamp":"2024-03-14T22:05:34.295063+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.295063+0000","objects_scrubbed":1,"log_size":9353,"log_dups_size":0,"ondisk_log_size":9353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:46.557728+0000","scrub_duration":0.019716635999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9374,"num_write_kb":71713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'10198","reported_seq":10249,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:55.934953+0000","last_change":"2024-03-14T22:05:55.935083+0000","last_active":"2024-03-14T22:05:55.934953+0000","last_peered":"2024-03-14T22:05:55.934953+0000","last_clean":"2024-03-14T22:05:55.934953+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T22:05:55.934953+0000","last_undegraded":"2024-03-14T22:05:55.934953+0000","last_fullsized":"2024-03-14T22:05:55.934953+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10198","last_scrub_stamp":"2024-03-14T22:05:55.934931+0000","last_deep_scrub":"23'10198","last_deep_scrub_stamp":"2024-03-14T22:05:55.934931+0000","last_clean_scrub_stamp":"2024-03-14T22:05:55.934931+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:55.234216+0000","scrub_duration":0.010658134999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8519,"num_write_kb":106953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'11405","reported_seq":11854,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:46.439556+0000","last_change":"2024-03-14T22:05:46.439556+0000","last_active":"2024-03-14T22:05:46.439556+0000","last_peered":"2024-03-14T22:05:46.439556+0000","last_clean":"2024-03-14T22:05:46.439556+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T22:05:46.439556+0000","last_undegraded":"2024-03-14T22:05:46.439556+0000","last_fullsized":"2024-03-14T22:05:46.439556+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11405","last_scrub_stamp":"2024-03-14T22:05:46.439398+0000","last_deep_scrub":"23'11405","last_deep_scrub_stamp":"2024-03-14T22:05:46.439398+0000","last_clean_scrub_stamp":"2024-03-14T22:05:46.439398+0000","objects_scrubbed":5,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:17.165906+0000","scrub_duration":0.054388652000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13689588,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":402,"num_read_kb":1363,"num_write":11407,"num_write_kb":86544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":368,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'13463","reported_seq":13505,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:25.800968+0000","last_change":"2024-03-14T21:14:03.040253+0000","last_active":"2024-03-14T22:05:25.800968+0000","last_peered":"2024-03-14T22:05:25.800968+0000","last_clean":"2024-03-14T22:05:25.800968+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T22:05:25.800968+0000","last_undegraded":"2024-03-14T22:05:25.800968+0000","last_fullsized":"2024-03-14T22:05:25.800968+0000","mapping_epoch":19,"log_start":"23'3400","ondisk_log_start":"23'3400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T21:14:02.638852+0000","last_clean_scrub_stamp":"2024-03-14T21:14:02.638852+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":11847,"num_write_kb":149794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'10677","reported_seq":10722,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:54.646809+0000","last_change":"2024-03-14T22:05:54.646938+0000","last_active":"2024-03-14T22:05:54.646809+0000","last_peered":"2024-03-14T22:05:54.646809+0000","last_clean":"2024-03-14T22:05:54.646809+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T22:05:54.646809+0000","last_undegraded":"2024-03-14T22:05:54.646809+0000","last_fullsized":"2024-03-14T22:05:54.646809+0000","mapping_epoch":19,"log_start":"23'600","ondisk_log_start":"23'600","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10677","last_scrub_stamp":"2024-03-14T22:05:54.646787+0000","last_deep_scrub":"23'10677","last_deep_scrub_stamp":"2024-03-14T22:05:54.646787+0000","last_clean_scrub_stamp":"2024-03-14T22:05:54.646787+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:23:14.680060+0000","scrub_duration":0.011557494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":8986,"num_write_kb":111494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6869","reported_seq":6914,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:45.414748+0000","last_change":"2024-03-14T22:05:45.414869+0000","last_active":"2024-03-14T22:05:45.414748+0000","last_peered":"2024-03-14T22:05:45.414748+0000","last_clean":"2024-03-14T22:05:45.414748+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T22:05:45.414748+0000","last_undegraded":"2024-03-14T22:05:45.414748+0000","last_fullsized":"2024-03-14T22:05:45.414748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6869","last_scrub_stamp":"2024-03-14T22:05:45.414726+0000","last_deep_scrub":"23'6869","last_deep_scrub_stamp":"2024-03-14T22:05:45.414726+0000","last_clean_scrub_stamp":"2024-03-14T22:05:45.414726+0000","objects_scrubbed":5,"log_size":6869,"log_dups_size":0,"ondisk_log_size":6869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:27.443736+0000","scrub_duration":0.057950375999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6885,"num_write_kb":66562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'11249","reported_seq":11298,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:48.149731+0000","last_change":"2024-03-14T22:05:48.149866+0000","last_active":"2024-03-14T22:05:48.149731+0000","last_peered":"2024-03-14T22:05:48.149731+0000","last_clean":"2024-03-14T22:05:48.149731+0000","last_became_active":"2024-03-14T21:14:44.859128+0000","last_became_peered":"2024-03-14T21:14:44.859128+0000","last_unstale":"2024-03-14T22:05:48.149731+0000","last_undegraded":"2024-03-14T22:05:48.149731+0000","last_fullsized":"2024-03-14T22:05:48.149731+0000","mapping_epoch":22,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11249","last_scrub_stamp":"2024-03-14T22:05:48.149707+0000","last_deep_scrub":"23'11249","last_deep_scrub_stamp":"2024-03-14T22:05:48.149707+0000","last_clean_scrub_stamp":"2024-03-14T22:05:48.149707+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:38.172409+0000","scrub_duration":0.015019927000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1780,"num_write":9572,"num_write_kb":116659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15428","reported_seq":16542,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:57.600392+0000","last_change":"2024-03-14T22:05:57.600521+0000","last_active":"2024-03-14T22:05:57.600392+0000","last_peered":"2024-03-14T22:05:57.600392+0000","last_clean":"2024-03-14T22:05:57.600392+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T22:05:57.600392+0000","last_undegraded":"2024-03-14T22:05:57.600392+0000","last_fullsized":"2024-03-14T22:05:57.600392+0000","mapping_epoch":18,"log_start":"23'5400","ondisk_log_start":"23'5400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'15428","last_scrub_stamp":"2024-03-14T22:05:57.600372+0000","last_deep_scrub":"23'15428","last_deep_scrub_stamp":"2024-03-14T22:05:57.600372+0000","last_clean_scrub_stamp":"2024-03-14T22:05:57.600372+0000","objects_scrubbed":6,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:50.979369+0000","scrub_duration":0.079319829999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1069,"num_read_kb":4840,"num_write":15404,"num_write_kb":112501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'10432","reported_seq":10492,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:06.161157+0000","last_change":"2024-03-14T22:06:06.161157+0000","last_active":"2024-03-14T22:06:06.161157+0000","last_peered":"2024-03-14T22:06:06.161157+0000","last_clean":"2024-03-14T22:06:06.161157+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T22:06:06.161157+0000","last_undegraded":"2024-03-14T22:06:06.161157+0000","last_fullsized":"2024-03-14T22:06:06.161157+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10432","last_scrub_stamp":"2024-03-14T22:06:06.160892+0000","last_deep_scrub":"23'10432","last_deep_scrub_stamp":"2024-03-14T22:06:06.160892+0000","last_clean_scrub_stamp":"2024-03-14T22:06:06.160892+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:59.539890+0000","scrub_duration":0.005531083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2796,"num_write":8857,"num_write_kb":109660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'8707","reported_seq":8757,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:33.083322+0000","last_change":"2024-03-14T22:05:33.083418+0000","last_active":"2024-03-14T22:05:33.083322+0000","last_peered":"2024-03-14T22:05:33.083322+0000","last_clean":"2024-03-14T22:05:33.083322+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T22:05:33.083322+0000","last_undegraded":"2024-03-14T22:05:33.083322+0000","last_fullsized":"2024-03-14T22:05:33.083322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8707","last_scrub_stamp":"2024-03-14T22:05:33.083300+0000","last_deep_scrub":"23'8707","last_deep_scrub_stamp":"2024-03-14T22:05:33.083300+0000","last_clean_scrub_stamp":"2024-03-14T22:05:33.083300+0000","objects_scrubbed":1,"log_size":8707,"log_dups_size":0,"ondisk_log_size":8707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:45.345964+0000","scrub_duration":0.015359457999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8698,"num_write_kb":67327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'11837","reported_seq":11887,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:02.951947+0000","last_change":"2024-03-14T22:06:02.952055+0000","last_active":"2024-03-14T22:06:02.951947+0000","last_peered":"2024-03-14T22:06:02.951947+0000","last_clean":"2024-03-14T22:06:02.951947+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T22:06:02.951947+0000","last_undegraded":"2024-03-14T22:06:02.951947+0000","last_fullsized":"2024-03-14T22:06:02.951947+0000","mapping_epoch":19,"log_start":"23'1800","ondisk_log_start":"23'1800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11837","last_scrub_stamp":"2024-03-14T22:06:02.951924+0000","last_deep_scrub":"23'11837","last_deep_scrub_stamp":"2024-03-14T22:06:02.951924+0000","last_clean_scrub_stamp":"2024-03-14T22:06:02.951924+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:21.687175+0000","scrub_duration":0.01245275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":10179,"num_write_kb":107836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6051","reported_seq":6101,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.157143+0000","last_change":"2024-03-14T22:05:31.157143+0000","last_active":"2024-03-14T22:05:31.157143+0000","last_peered":"2024-03-14T22:05:31.157143+0000","last_clean":"2024-03-14T22:05:31.157143+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T22:05:31.157143+0000","last_undegraded":"2024-03-14T22:05:31.157143+0000","last_fullsized":"2024-03-14T22:05:31.157143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6051","last_scrub_stamp":"2024-03-14T22:05:31.157033+0000","last_deep_scrub":"23'6051","last_deep_scrub_stamp":"2024-03-14T22:05:31.157033+0000","last_clean_scrub_stamp":"2024-03-14T22:05:31.157033+0000","objects_scrubbed":5,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:11:59.042596+0000","scrub_duration":0.052529798000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6048,"num_write_kb":48854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":345,"num_read_kb":87814,"num_write":552757,"num_write_kb":7123402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624278,"ondisk_log_size":624278,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":256907613,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":3941,"num_read_kb":24159,"num_write":290960,"num_write_kb":2282127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23590,"num_omap_keys":34,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":513884160,"data_stored":513815226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271687,"ondisk_log_size":271687,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575480,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252664,"kb_used_data":112496,"kb_used_omap":1,"kb_used_meta":1140158,"kb_avail":93119176,"statfs":{"total":96636764160,"available":95354036224,"internally_reserved":0,"allocated":115195904,"data_stored":115031593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1167522059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44600000000000001,"15min":0.48899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.19500000000000001},"max":{"1min":0.879,"5min":2.782,"15min":4.7999999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.63100000000000001,"15min":0.53200000000000003},"min":{"1min":0.29199999999999998,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":4.8860000000000001,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.51300000000000001,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":4.8410000000000002,"5min":4.8410000000000002,"15min":4.8410000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.28599999999999998,"15min":0.26400000000000001},"max":{"1min":2.2240000000000002,"5min":6.1440000000000001,"15min":6.1440000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52500000000000002,"15min":0.53200000000000003},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.25900000000000001},"max":{"1min":0.80300000000000005,"5min":3.8210000000000002,"15min":4.8310000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.63800000000000001,"15min":0.55300000000000005},"min":{"1min":0.36199999999999999,"5min":0.309,"15min":0.25900000000000001},"max":{"1min":4.1669999999999998,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.2519999999999998,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.629},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":3.165,"5min":3.165,"15min":4.851},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.254,"5min":0.23400000000000001,"15min":0.22500000000000001},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":5.0590000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.41399999999999998,"15min":0.44900000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.23300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":5.2679999999999998},"last":0.745}]},{"osd":5,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.878,"5min":1.3089999999999999,"15min":3.5760000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.51800000000000002,"15min":0.54500000000000004},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":6.4790000000000001},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":1.153,"5min":1.2529999999999999,"15min":4.157},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.29299999999999998},"max":{"1min":2.403,"5min":2.403,"15min":5.0880000000000001},"last":0.77200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575479,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101820,"kb_used_data":69876,"kb_used_omap":1,"kb_used_meta":1031934,"kb_avail":93270020,"statfs":{"total":96636764160,"available":95508500480,"internally_reserved":0,"allocated":71553024,"data_stored":71391599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1056700683},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.47299999999999998,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.188},"max":{"1min":2.899,"5min":2.899,"15min":17.885999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.23000000000000001},"max":{"1min":6.1449999999999996,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.498}]},{"osd":1,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.52000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.216},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":7.226},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.27800000000000002,"5min":0.24099999999999999,"15min":0.23699999999999999},"max":{"1min":0.75700000000000001,"5min":1.224,"15min":7.1699999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.6189999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.34399999999999997,"5min":0.30599999999999999,"15min":0.26800000000000002},"max":{"1min":3.282,"5min":3.282,"15min":7.2489999999999997},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52300000000000002,"15min":0.55800000000000005},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":1.2470000000000001,"15min":7.1470000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.61799999999999999,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.32700000000000001,"15min":0.29899999999999999},"max":{"1min":6.1660000000000004,"5min":6.1660000000000004,"15min":6.1660000000000004},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.42999999999999999},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.468,"5min":2.468,"15min":7.3330000000000002},"last":0.373},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.49199999999999999,"15min":0.45300000000000001},"min":{"1min":0.26200000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":6.0229999999999997,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":8.234,"5min":8.234,"15min":8.234},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48799999999999999,"15min":0.503},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.26900000000000002},"max":{"1min":0.90600000000000003,"5min":1.3340000000000001,"15min":7.5380000000000003},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Mar 14 22:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.54100000000000004,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.26400000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":7.2039999999999997},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575479,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1151736,"kb_used_data":69872,"kb_used_omap":1,"kb_used_meta":1081854,"kb_avail":93220104,"statfs":{"total":96636764160,"available":95457386496,"internally_reserved":0,"allocated":71548928,"data_stored":71391017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1107818763},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.54300000000000004},"min":{"1min":0.26200000000000001,"5min":0.223,"15min":0.19400000000000001},"max":{"1min":0.66100000000000003,"5min":3.2149999999999999,"15min":7.0990000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.53000000000000003,"15min":0.57299999999999995},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.21099999999999999},"max":{"1min":0.73799999999999999,"5min":5.7359999999999998,"15min":12.281000000000001},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.68799999999999994,"15min":0.57599999999999996},"min":{"1min":0.245,"5min":0.245,"15min":0.20599999999999999},"max":{"1min":0.63100000000000001,"5min":14.952,"15min":14.952},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.52200000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.247},"max":{"1min":0.745,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.75800000000000001,"15min":0.61599999999999999},"min":{"1min":0.29099999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":18.623000000000001,"15min":18.623000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.96099999999999997,"15min":0.67300000000000004},"min":{"1min":0.32200000000000001,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":30.004999999999999,"15min":30.004999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28000000000000003},"max":{"1min":0.69599999999999995,"5min":3.6600000000000001,"15min":9.2309999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":0.82699999999999996,"15min":7.3780000000000001},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.59899999999999998,"15min":0.50700000000000001},"min":{"1min":0.23599999999999999,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.51900000000000002,"5min":12.414,"15min":12.414},"last":0.375},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.50800000000000001,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.66900000000000004,"5min":3.1970000000000001,"15min":8.3339999999999996},"last":0.42699999999999999}]},{"osd":6,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45100000000000001,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.245,"15min":0.22},"max":{"1min":0.64000000000000001,"5min":1.905,"15min":7.4000000000000004},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.621,"15min":0.55400000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.24299999999999999},"max":{"1min":0.63200000000000001,"5min":10.278,"15min":10.278},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":0.68500000000000005,"5min":3.262,"15min":7.1619999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54700000000000004,"15min":0.56000000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":3.2909999999999999,"15min":7.3310000000000004},"last":0.39700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":965948,"kb_used_data":42292,"kb_used_omap":1,"kb_used_meta":923646,"kb_avail":93405892,"statfs":{"total":96636764160,"available":95647633408,"internally_reserved":0,"allocated":43307008,"data_stored":43137601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":945813771},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.502,"15min":0.56200000000000006},"min":{"1min":0.41099999999999998,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":1.1319999999999999,"5min":2.23,"15min":15.013},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.20499999999999999},"max":{"1min":1.3120000000000001,"5min":2.1139999999999999,"15min":5.6879999999999997},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.254},"max":{"1min":1.3500000000000001,"5min":1.806,"15min":4.6479999999999997},"last":0.747},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.48799999999999999,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":4.0469999999999997},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.29099999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":5.5540000000000003},"last":0.628},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34399999999999997,"15min":0.29599999999999999},"max":{"1min":1.379,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":9.9800000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.899},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.441,"15min":0.41899999999999998},"min":{"1min":0.25,"5min":0.219,"15min":0.219},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.0470000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.22700000000000001},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.501}]},{"osd":6,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.245},"max":{"1min":2.21,"5min":2.21,"15min":3.8650000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.495,"15min":0.46899999999999997},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.24299999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.501,"15min":0.505},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23400000000000001},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":8.7639999999999993},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52700000000000002,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":3.7850000000000001},"last":0.56599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1135420,"kb_used_data":57588,"kb_used_omap":1,"kb_used_meta":1077822,"kb_avail":93236420,"statfs":{"total":96636764160,"available":95474094080,"internally_reserved":0,"allocated":58970112,"data_stored":58808687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1103689995},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.17000000000000001},"max":{"1min":8.3350000000000009,"5min":8.3350000000000009,"15min":14.695},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.41699999999999998,"15min":0.52100000000000002},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.186},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":8.9649999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.41299999999999998,"15min":0.46000000000000002},"min":{"1min":0.22,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":5.5289999999999999},"last":0.443},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.55400000000000005,"15min":0.59199999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.20100000000000001},"max":{"1min":1.508,"5min":6.1740000000000004,"15min":14.898},"last":0.498}]},{"osd":2,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46300000000000002,"15min":0.56799999999999995},"min":{"1min":0.26300000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61599999999999999,"5min":3.7869999999999999,"15min":9.5150000000000006},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.28899999999999998,"5min":0.25800000000000001,"15min":0.23100000000000001},"max":{"1min":2.2090000000000001,"5min":2.9430000000000001,"15min":5.2859999999999996},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.74399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":0.84299999999999997,"5min":1.5229999999999999,"15min":14.071999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.60699999999999998,"15min":0.69899999999999995},"min":{"1min":0.40000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":14.109},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64400000000000002,"15min":0.68100000000000005},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.314},"max":{"1min":3.0209999999999999,"5min":5.1609999999999996,"15min":11.443},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.27800000000000002},"max":{"1min":8.3659999999999997,"5min":8.3659999999999997,"15min":14.550000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65600000000000003,"15min":0.751},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.29299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2539999999999996,"15min":9.7200000000000006},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.65700000000000003,"15min":0.70699999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35599999999999998},"max":{"1min":2.6360000000000001,"5min":2.8170000000000002,"15min":15.339},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":8.2799999999999994,"5min":8.2799999999999994,"15min":14.726000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64200000000000002,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.33700000000000002},"max":{"1min":1.1180000000000001,"5min":2.9489999999999998,"15min":9.5079999999999991},"last":0.74299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575480,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649796,"kb_used_data":45244,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":93722044,"statfs":{"total":96636764160,"available":95971373056,"internally_reserved":0,"allocated":46329856,"data_stored":46172026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":619051275},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.58199999999999996,"15min":0.497},"min":{"1min":0.217,"5min":0.185,"15min":0.182},"max":{"1min":6.3840000000000003,"5min":7.008,"15min":10.869},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.56799999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":3.8170000000000002,"5min":7.7699999999999996,"15min":12.317},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.59699999999999998,"15min":0.49099999999999999},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":6.4249999999999998,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.20100000000000001},"max":{"1min":3.8769999999999998,"5min":7.8090000000000002,"15min":9.5429999999999993},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.63100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":2.5419999999999998,"5min":13.130000000000001,"15min":27.355},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.214,"5min":0.64700000000000002,"15min":0.62},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27500000000000002},"max":{"1min":7.5910000000000002,"5min":7.5910000000000002,"15min":12.451000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":0.90900000000000003,"15min":0.77200000000000002},"min":{"1min":0.46400000000000002,"5min":0.32100000000000001,"15min":0.28899999999999998},"max":{"1min":7.6239999999999997,"5min":8.0440000000000005,"15min":12.478999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.3289999999999997,"5min":7.9409999999999998,"15min":12.391999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.90200000000000002,"15min":0.76700000000000002},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":7.4089999999999998,"5min":13.318,"15min":20.84},"last":0.503},{"interface":"front","average":{"1min":1.5549999999999999,"5min":0.79300000000000004,"15min":0.73699999999999999},"min":{"1min":0.40799999999999997,"5min":0.35799999999999998,"15min":0.32000000000000001},"max":{"1min":12.319000000000001,"5min":12.319000000000001,"15min":12.319000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.65400000000000003,"15min":0.66100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34300000000000003},"max":{"1min":2.6429999999999998,"5min":5.0209999999999999,"15min":6.2560000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.05,"5min":0.84899999999999998,"15min":0.68700000000000006},"min":{"1min":0.40100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.4420000000000002,"5min":13.177,"15min":13.177},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.74299999999999999,"15min":0.71899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.32600000000000001},"max":{"1min":3.8490000000000002,"5min":4.9800000000000004,"15min":6.2309999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.504,"5min":3.5830000000000002,"15min":6.4279999999999999},"last":0.65500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575479,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1031244,"kb_used_data":57540,"kb_used_omap":1,"kb_used_meta":973694,"kb_avail":93340596,"statfs":{"total":96636764160,"available":95580770304,"internally_reserved":0,"allocated":58920960,"data_stored":58755098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":997062923},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.186},"max":{"1min":1.7090000000000001,"5min":3.625,"15min":10.891999999999999},"last":0.377},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46999999999999997,"15min":0.50700000000000001},"min":{"1min":0.24399999999999999,"5min":0.219,"15min":0.19700000000000001},"max":{"1min":1.341,"5min":6.5339999999999998,"15min":9.2249999999999996},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.49199999999999999},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.20999999999999999},"max":{"1min":2.6589999999999998,"5min":13.553000000000001,"15min":13.553000000000001},"last":0.504},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50700000000000001,"15min":0.53400000000000003},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.23300000000000001},"max":{"1min":3.008,"5min":6.6050000000000004,"15min":7.1180000000000003},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.307,"5min":0.251,"15min":0.22900000000000001},"max":{"1min":1.3120000000000001,"5min":10.236000000000001,"15min":24.879000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.53800000000000003,"15min":0.57199999999999995},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.22900000000000001},"max":{"1min":10.433,"5min":10.433,"15min":20.995000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":1.748,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":2.9390000000000001,"5min":8.4420000000000002,"15min":8.4420000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.621,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.3940000000000001,"5min":6.3940000000000001,"15min":8.6419999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80200000000000005,"15min":0.70299999999999996},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.8100000000000001,"5min":16.943999999999999,"15min":16.943999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.32100000000000001},"max":{"1min":2.379,"5min":2.379,"15min":7.468},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.33600000000000002},"max":{"1min":2.3290000000000002,"5min":3.4950000000000001,"15min":5.5030000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":2.9780000000000002,"5min":6.5720000000000001,"15min":9.3209999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.36099999999999999},"max":{"1min":1.52,"5min":3.8959999999999999,"15min":10.346},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":871104,"kb_used_data":49976,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":93500736,"statfs":{"total":96636764160,"available":95744753664,"internally_reserved":0,"allocated":51175424,"data_stored":51009794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":840825099},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.45500000000000002},"min":{"1min":0.214,"5min":0.19800000000000001,"15min":0.17499999999999999},"max":{"1min":2.8149999999999999,"5min":12.734,"15min":12.734},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.52000000000000002,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":3.976,"5min":9.3620000000000001,"15min":9.3620000000000001},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.843,"5min":15.375,"15min":15.375},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.48099999999999998},"min":{"1min":0.28399999999999997,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.008,"5min":9.2959999999999994,"15min":9.2959999999999994},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.45800000000000002},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.20699999999999999},"max":{"1min":2.7040000000000002,"5min":8.7230000000000008,"15min":8.7230000000000008},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.55300000000000005},"min":{"1min":0.26300000000000001,"5min":0.245,"15min":0.245},"max":{"1min":2.9169999999999998,"5min":10.928000000000001,"15min":12.864000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.0350000000000001,"5min":9.2319999999999993,"15min":9.2319999999999993},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":3.7930000000000001,"15min":7.5499999999999998},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.61899999999999999},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":3.0790000000000002,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73899999999999999,"15min":0.68700000000000006},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33700000000000002},"max":{"1min":2.8660000000000001,"5min":10.135999999999999,"15min":11.068},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77400000000000002,"15min":0.69699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.8959999999999999,"5min":12.77,"15min":12.77},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.66200000000000003},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":4.0579999999999998,"5min":9.391,"15min":9.391},"last":0.499}]},{"osd":7,"last update":"Thu Mar 14 22:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73299999999999998,"15min":0.67900000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.0830000000000002,"5min":9.3290000000000006,"15min":9.3290000000000006},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.71499999999999997,"15min":0.65400000000000003},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.33300000000000002},"max":{"1min":0.78900000000000003,"5min":7.8739999999999997,"15min":7.8739999999999997},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58679296,"data_stored":58667339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":46088192,"data_stored":46084267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43065344,"data_stored":43049842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":71307264,"data_stored":71303258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":71311360,"data_stored":71303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114360320,"data_stored":114353466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T22:06:13.432 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.433 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.433 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.433 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.433 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.433 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.433 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.434 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.434 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.434 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-14T21:14:02.638852+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=21, tm_min=14, tm_sec=2, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=22, tm_min=5, tm_sec=27, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T22:06:13.435 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-14T22:06:23.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:23.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:23.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:33.435 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T22:06:33.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.554+0000 7f5327b11640 1 Processor -- start 2024-03-14T22:06:33.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.554+0000 7f5327b11640 1 -- start start 2024-03-14T22:06:33.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f5327b11640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53201480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f5327b11640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f532006b620 0x7f53200bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f5327b11640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532006a6e0 con 0x7f53200bc940 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f5327b11640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532006ad10 con 0x7f532006b620 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f5327b11640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200bc2b0 con 0x7f53200baef0 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f53250ac640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f532006b620 0x7f53200bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f53250ac640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f532006b620 0x7f53200bf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50158/0 (socket says 172.21.15.49:50158) 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f53250ac640 1 -- 172.21.15.49:0/3361011892 learned_addr learned my addr 172.21.15.49:0/3361011892 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f53258ad640 1 --2- 172.21.15.49:0/3361011892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53201480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f53250ac640 1 -- 172.21.15.49:0/3361011892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 msgr2=0x7f53201480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f53250ac640 1 --2- 172.21.15.49:0/3361011892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53201480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.555+0000 7f53250ac640 1 -- 172.21.15.49:0/3361011892 >> v1:172.21.15.97:6789/0 conn(0x7f53200bc940 legacy=0x7f532014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.556+0000 7f53250ac640 1 -- 172.21.15.49:0/3361011892 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532006bab0 con 0x7f532006b620 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.556+0000 7f53250ac640 1 --2- 172.21.15.49:0/3361011892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f532006b620 0x7f53200bf7f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5310009870 tx=0x7f531002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47522c722a807eb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.556+0000 7f53248ab640 1 -- 172.21.15.49:0/3361011892 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310031640 con 0x7f532006b620 2024-03-14T22:06:33.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.556+0000 7f53248ab640 1 -- 172.21.15.49:0/3361011892 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53100317a0 con 0x7f532006b620 2024-03-14T22:06:33.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.556+0000 7f53248ab640 1 -- 172.21.15.49:0/3361011892 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310031b30 con 0x7f532006b620 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 -- 172.21.15.49:0/3361011892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f532006b620 msgr2=0x7f53200bf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 --2- 172.21.15.49:0/3361011892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f532006b620 0x7f53200bf7f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5310009870 tx=0x7f531002b8f0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 -- 172.21.15.49:0/3361011892 shutdown_connections 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 --2- 172.21.15.49:0/3361011892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f532006b620 0x7f53200bf7f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 --2- 172.21.15.49:0/3361011892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53201480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 -- 172.21.15.49:0/3361011892 >> 172.21.15.49:0/3361011892 conn(0x7f53200bcd80 msgr2=0x7f53200bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 -- 172.21.15.49:0/3361011892 shutdown_connections 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.557+0000 7f5327b11640 1 -- 172.21.15.49:0/3361011892 wait complete. 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.558+0000 7f5327b11640 1 Processor -- start 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.558+0000 7f5327b11640 1 -- start start 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f5327b11640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53200be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f5327b11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53200bc940 0x7f53200a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f5327b11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53201dfd30 0x7f53200b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f5327b11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200bee30 con 0x7f53200bc940 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f5327b11640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201e0d10 con 0x7f53201dfd30 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f5327b11640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200a21b0 con 0x7f53200baef0 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53200bc940 0x7f53200a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53200bc940 0x7f53200a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50164/0 (socket says 172.21.15.49:50164) 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 -- 172.21.15.49:0/25740483 learned_addr learned my addr 172.21.15.49:0/25740483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53258ad640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53200be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53260ae640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53201dfd30 0x7f53200b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 -- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 msgr2=0x7f53200be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53200be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 -- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53201dfd30 msgr2=0x7f53200b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53201dfd30 0x7f53200b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.559+0000 7f53250ac640 1 -- 172.21.15.49:0/25740483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53100094d0 con 0x7f53200bc940 2024-03-14T22:06:33.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.560+0000 7f53250ac640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53200bc940 0x7f53200a1c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5310000c00 tx=0x7f53100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:33.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.560+0000 7f53167fc640 1 -- 172.21.15.49:0/25740483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531003a930 con 0x7f53200bc940 2024-03-14T22:06:33.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.560+0000 7f53167fc640 1 -- 172.21.15.49:0/25740483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531003aa90 con 0x7f53200bc940 2024-03-14T22:06:33.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.560+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53200ba600 con 0x7f53200bc940 2024-03-14T22:06:33.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.560+0000 7f53167fc640 1 -- 172.21.15.49:0/25740483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531003ae40 con 0x7f53200bc940 2024-03-14T22:06:33.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.560+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53200a2420 con 0x7f53200bc940 2024-03-14T22:06:33.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.561+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53200666a0 con 0x7f53200bc940 2024-03-14T22:06:33.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.561+0000 7f53167fc640 1 -- 172.21.15.49:0/25740483 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f531002f050 con 0x7f53200bc940 2024-03-14T22:06:33.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.562+0000 7f53167fc640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f530406a7a0 0x7f530406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.562+0000 7f53167fc640 1 -- 172.21.15.49:0/25740483 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f53100aab80 con 0x7f53200bc940 2024-03-14T22:06:33.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.562+0000 7f53258ad640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f530406a7a0 0x7f530406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.562+0000 7f53258ad640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f530406a7a0 0x7f530406cc60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f531c009b40 tx=0x7f531c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:33.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.566+0000 7f53167fc640 1 -- 172.21.15.49:0/25740483 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5310075fd0 con 0x7f53200bc940 2024-03-14T22:06:33.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.741+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 --> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53200674d0 con 0x7f530406a7a0 2024-03-14T22:06:33.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.749+0000 7f53167fc640 1 -- 172.21.15.49:0/25740483 <== mgr.4108 v2:172.21.15.97:6832/37329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294197 (secure 0 0 0) 0x7f53200674d0 con 0x7f530406a7a0 2024-03-14T22:06:33.752 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:06:33.754 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-14T22:06:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f530406a7a0 msgr2=0x7f530406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f530406a7a0 0x7f530406cc60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f531c009b40 tx=0x7f531c009190 comp rx=0 tx=0).stop 2024-03-14T22:06:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53200bc940 msgr2=0x7f53200a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53200bc940 0x7f53200a1c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5310000c00 tx=0x7f53100046c0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 shutdown_connections 2024-03-14T22:06:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f530406a7a0 0x7f530406cc60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53201dfd30 0x7f53200b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53200bc940 0x7f53200a1c70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 --2- 172.21.15.49:0/25740483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53200baef0 0x7f53200be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 >> 172.21.15.49:0/25740483 conn(0x7f53200bcd80 msgr2=0x7f5320145d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 shutdown_connections 2024-03-14T22:06:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.752+0000 7f5327b11640 1 -- 172.21.15.49:0/25740483 wait complete. 2024-03-14T22:06:33.763 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":1598,"stamp":"2024-03-14T22:06:33.606377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257497981,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":4402,"num_read_kb":112073,"num_write":843876,"num_write_kb":9407945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23590,"num_omap_keys":34,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":896084,"ondisk_log_size":896084,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8159732,"kb_used_data":504884,"kb_used_omap":13,"kb_used_meta":7654770,"kb_avail":746814988,"statfs":{"total":773094113280,"available":764738547712,"internally_reserved":0,"allocated":517001216,"data_stored":515697415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":7838484568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003474"},"pg_stats":[{"pgid":"3.3f","version":"23'10331","reported_seq":10386,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:21.564080+0000","last_change":"2024-03-14T22:06:21.564193+0000","last_active":"2024-03-14T22:06:21.564080+0000","last_peered":"2024-03-14T22:06:21.564080+0000","last_clean":"2024-03-14T22:06:21.564080+0000","last_became_active":"2024-03-14T21:14:03.030002+0000","last_became_peered":"2024-03-14T21:14:03.030002+0000","last_unstale":"2024-03-14T22:06:21.564080+0000","last_undegraded":"2024-03-14T22:06:21.564080+0000","last_fullsized":"2024-03-14T22:06:21.564080+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10331","last_scrub_stamp":"2024-03-14T22:06:21.564057+0000","last_deep_scrub":"23'10331","last_deep_scrub_stamp":"2024-03-14T22:06:21.564057+0000","last_clean_scrub_stamp":"2024-03-14T22:06:21.564057+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T10:00:20.198706+0000","scrub_duration":0.0048552450000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8785,"num_write_kb":106962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'9098","reported_seq":9160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:04.441648+0000","last_change":"2024-03-14T22:06:04.441648+0000","last_active":"2024-03-14T22:06:04.441648+0000","last_peered":"2024-03-14T22:06:04.441648+0000","last_clean":"2024-03-14T22:06:04.441648+0000","last_became_active":"2024-03-14T21:14:03.030897+0000","last_became_peered":"2024-03-14T21:14:03.030897+0000","last_unstale":"2024-03-14T22:06:04.441648+0000","last_undegraded":"2024-03-14T22:06:04.441648+0000","last_fullsized":"2024-03-14T22:06:04.441648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9098","last_scrub_stamp":"2024-03-14T22:06:04.441479+0000","last_deep_scrub":"23'9098","last_deep_scrub_stamp":"2024-03-14T22:06:04.441479+0000","last_clean_scrub_stamp":"2024-03-14T22:06:04.441479+0000","objects_scrubbed":0,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:23.176730+0000","scrub_duration":0.011687632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":7584,"num_write_kb":103945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'11564","reported_seq":11618,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:08.135625+0000","last_change":"2024-03-14T22:06:08.135625+0000","last_active":"2024-03-14T22:06:08.135625+0000","last_peered":"2024-03-14T22:06:08.135625+0000","last_clean":"2024-03-14T22:06:08.135625+0000","last_became_active":"2024-03-14T21:14:03.034153+0000","last_became_peered":"2024-03-14T21:14:03.034153+0000","last_unstale":"2024-03-14T22:06:08.135625+0000","last_undegraded":"2024-03-14T22:06:08.135625+0000","last_fullsized":"2024-03-14T22:06:08.135625+0000","mapping_epoch":19,"log_start":"23'1500","ondisk_log_start":"23'1500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11564","last_scrub_stamp":"2024-03-14T22:06:08.135455+0000","last_deep_scrub":"23'11564","last_deep_scrub_stamp":"2024-03-14T22:06:08.135455+0000","last_clean_scrub_stamp":"2024-03-14T22:06:08.135455+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:26.870706+0000","scrub_duration":0.011002987000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1274,"num_write":9914,"num_write_kb":127294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'10102","reported_seq":10161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:09.658830+0000","last_change":"2024-03-14T22:06:09.658830+0000","last_active":"2024-03-14T22:06:09.658830+0000","last_peered":"2024-03-14T22:06:09.658830+0000","last_clean":"2024-03-14T22:06:09.658830+0000","last_became_active":"2024-03-14T21:14:03.034323+0000","last_became_peered":"2024-03-14T21:14:03.034323+0000","last_unstale":"2024-03-14T22:06:09.658830+0000","last_undegraded":"2024-03-14T22:06:09.658830+0000","last_fullsized":"2024-03-14T22:06:09.658830+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10102","last_scrub_stamp":"2024-03-14T22:06:09.658681+0000","last_deep_scrub":"23'10102","last_deep_scrub_stamp":"2024-03-14T22:06:09.658681+0000","last_clean_scrub_stamp":"2024-03-14T22:06:09.658681+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:25:59.909681+0000","scrub_duration":0.014752894000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8407,"num_write_kb":110879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'9608","reported_seq":9654,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:18.546710+0000","last_change":"2024-03-14T22:06:18.546827+0000","last_active":"2024-03-14T22:06:18.546710+0000","last_peered":"2024-03-14T22:06:18.546710+0000","last_clean":"2024-03-14T22:06:18.546710+0000","last_became_active":"2024-03-14T21:14:03.033228+0000","last_became_peered":"2024-03-14T21:14:03.033228+0000","last_unstale":"2024-03-14T22:06:18.546710+0000","last_undegraded":"2024-03-14T22:06:18.546710+0000","last_fullsized":"2024-03-14T22:06:18.546710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9608","last_scrub_stamp":"2024-03-14T22:06:18.546686+0000","last_deep_scrub":"23'9608","last_deep_scrub_stamp":"2024-03-14T22:06:18.546686+0000","last_clean_scrub_stamp":"2024-03-14T22:06:18.546686+0000","objects_scrubbed":0,"log_size":9608,"log_dups_size":0,"ondisk_log_size":9608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:48:00.547956+0000","scrub_duration":0.010707978999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8008,"num_write_kb":102192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'9645","reported_seq":9697,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:10.665830+0000","last_change":"2024-03-14T22:06:10.665963+0000","last_active":"2024-03-14T22:06:10.665830+0000","last_peered":"2024-03-14T22:06:10.665830+0000","last_clean":"2024-03-14T22:06:10.665830+0000","last_became_active":"2024-03-14T21:14:03.032147+0000","last_became_peered":"2024-03-14T21:14:03.032147+0000","last_unstale":"2024-03-14T22:06:10.665830+0000","last_undegraded":"2024-03-14T22:06:10.665830+0000","last_fullsized":"2024-03-14T22:06:10.665830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9645","last_scrub_stamp":"2024-03-14T22:06:10.665805+0000","last_deep_scrub":"23'9645","last_deep_scrub_stamp":"2024-03-14T22:06:10.665805+0000","last_clean_scrub_stamp":"2024-03-14T22:06:10.665805+0000","objects_scrubbed":0,"log_size":9645,"log_dups_size":0,"ondisk_log_size":9645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T02:47:52.667075+0000","scrub_duration":0.012453291999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":8054,"num_write_kb":114907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'10037","reported_seq":10085,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:17.523365+0000","last_change":"2024-03-14T22:06:17.523509+0000","last_active":"2024-03-14T22:06:17.523365+0000","last_peered":"2024-03-14T22:06:17.523365+0000","last_clean":"2024-03-14T22:06:17.523365+0000","last_became_active":"2024-03-14T21:14:03.030318+0000","last_became_peered":"2024-03-14T21:14:03.030318+0000","last_unstale":"2024-03-14T22:06:17.523365+0000","last_undegraded":"2024-03-14T22:06:17.523365+0000","last_fullsized":"2024-03-14T22:06:17.523365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10037","last_scrub_stamp":"2024-03-14T22:06:17.523343+0000","last_deep_scrub":"23'10037","last_deep_scrub_stamp":"2024-03-14T22:06:17.523343+0000","last_clean_scrub_stamp":"2024-03-14T22:06:17.523343+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:26:07.774342+0000","scrub_duration":0.011285478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":8423,"num_write_kb":104789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'11196","reported_seq":11248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:11.855181+0000","last_change":"2024-03-14T22:06:11.855300+0000","last_active":"2024-03-14T22:06:11.855181+0000","last_peered":"2024-03-14T22:06:11.855181+0000","last_clean":"2024-03-14T22:06:11.855181+0000","last_became_active":"2024-03-14T21:14:03.039114+0000","last_became_peered":"2024-03-14T21:14:03.039114+0000","last_unstale":"2024-03-14T22:06:11.855181+0000","last_undegraded":"2024-03-14T22:06:11.855181+0000","last_fullsized":"2024-03-14T22:06:11.855181+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11196","last_scrub_stamp":"2024-03-14T22:06:11.855159+0000","last_deep_scrub":"23'11196","last_deep_scrub_stamp":"2024-03-14T22:06:11.855159+0000","last_clean_scrub_stamp":"2024-03-14T22:06:11.855159+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:19:30.067015+0000","scrub_duration":0.014130953999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":9605,"num_write_kb":125652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'9017","reported_seq":9066,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:19.556144+0000","last_change":"2024-03-14T22:06:19.556248+0000","last_active":"2024-03-14T22:06:19.556144+0000","last_peered":"2024-03-14T22:06:19.556144+0000","last_clean":"2024-03-14T22:06:19.556144+0000","last_became_active":"2024-03-14T21:14:03.030261+0000","last_became_peered":"2024-03-14T21:14:03.030261+0000","last_unstale":"2024-03-14T22:06:19.556144+0000","last_undegraded":"2024-03-14T22:06:19.556144+0000","last_fullsized":"2024-03-14T22:06:19.556144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9017","last_scrub_stamp":"2024-03-14T22:06:19.556122+0000","last_deep_scrub":"23'9017","last_deep_scrub_stamp":"2024-03-14T22:06:19.556122+0000","last_clean_scrub_stamp":"2024-03-14T22:06:19.556122+0000","objects_scrubbed":0,"log_size":9017,"log_dups_size":0,"ondisk_log_size":9017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:22:57.498430+0000","scrub_duration":0.0087408190000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7381,"num_write_kb":104568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'9539","reported_seq":9587,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:11.756843+0000","last_change":"2024-03-14T22:06:11.756990+0000","last_active":"2024-03-14T22:06:11.756843+0000","last_peered":"2024-03-14T22:06:11.756843+0000","last_clean":"2024-03-14T22:06:11.756843+0000","last_became_active":"2024-03-14T21:14:03.032880+0000","last_became_peered":"2024-03-14T21:14:03.032880+0000","last_unstale":"2024-03-14T22:06:11.756843+0000","last_undegraded":"2024-03-14T22:06:11.756843+0000","last_fullsized":"2024-03-14T22:06:11.756843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9539","last_scrub_stamp":"2024-03-14T22:06:11.756820+0000","last_deep_scrub":"23'9539","last_deep_scrub_stamp":"2024-03-14T22:06:11.756820+0000","last_clean_scrub_stamp":"2024-03-14T22:06:11.756820+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:28:03.507110+0000","scrub_duration":0.01338114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7843,"num_write_kb":113016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'9229","reported_seq":9282,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.535716+0000","last_change":"2024-03-14T22:06:01.535842+0000","last_active":"2024-03-14T22:06:01.535716+0000","last_peered":"2024-03-14T22:06:01.535716+0000","last_clean":"2024-03-14T22:06:01.535716+0000","last_became_active":"2024-03-14T21:14:03.033361+0000","last_became_peered":"2024-03-14T21:14:03.033361+0000","last_unstale":"2024-03-14T22:06:01.535716+0000","last_undegraded":"2024-03-14T22:06:01.535716+0000","last_fullsized":"2024-03-14T22:06:01.535716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9229","last_scrub_stamp":"2024-03-14T22:06:01.535693+0000","last_deep_scrub":"23'9229","last_deep_scrub_stamp":"2024-03-14T22:06:01.535693+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.535693+0000","objects_scrubbed":0,"log_size":9229,"log_dups_size":0,"ondisk_log_size":9229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:00.834979+0000","scrub_duration":0.011388333,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7677,"num_write_kb":104837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'10311","reported_seq":10358,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:36.777685+0000","last_change":"2024-03-14T22:05:36.777822+0000","last_active":"2024-03-14T22:05:36.777685+0000","last_peered":"2024-03-14T22:05:36.777685+0000","last_clean":"2024-03-14T22:05:36.777685+0000","last_became_active":"2024-03-14T21:14:03.033639+0000","last_became_peered":"2024-03-14T21:14:03.033639+0000","last_unstale":"2024-03-14T22:05:36.777685+0000","last_undegraded":"2024-03-14T22:05:36.777685+0000","last_fullsized":"2024-03-14T22:05:36.777685+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10311","last_scrub_stamp":"2024-03-14T22:05:36.777662+0000","last_deep_scrub":"23'10311","last_deep_scrub_stamp":"2024-03-14T22:05:36.777662+0000","last_clean_scrub_stamp":"2024-03-14T22:05:36.777662+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:58:06.567940+0000","scrub_duration":0.019761048999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8730,"num_write_kb":105734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'8864","reported_seq":8914,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:59.996609+0000","last_change":"2024-03-14T22:05:59.996747+0000","last_active":"2024-03-14T22:05:59.996609+0000","last_peered":"2024-03-14T22:05:59.996609+0000","last_clean":"2024-03-14T22:05:59.996609+0000","last_became_active":"2024-03-14T21:14:03.041181+0000","last_became_peered":"2024-03-14T21:14:03.041181+0000","last_unstale":"2024-03-14T22:05:59.996609+0000","last_undegraded":"2024-03-14T22:05:59.996609+0000","last_fullsized":"2024-03-14T22:05:59.996609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8864","last_scrub_stamp":"2024-03-14T22:05:59.996584+0000","last_deep_scrub":"23'8864","last_deep_scrub_stamp":"2024-03-14T22:05:59.996584+0000","last_clean_scrub_stamp":"2024-03-14T22:05:59.996584+0000","objects_scrubbed":0,"log_size":8864,"log_dups_size":0,"ondisk_log_size":8864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:10.781627+0000","scrub_duration":0.0095402560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":7264,"num_write_kb":95419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'10416","reported_seq":10468,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:09.178102+0000","last_change":"2024-03-14T22:06:09.178225+0000","last_active":"2024-03-14T22:06:09.178102+0000","last_peered":"2024-03-14T22:06:09.178102+0000","last_clean":"2024-03-14T22:06:09.178102+0000","last_became_active":"2024-03-14T21:14:03.028019+0000","last_became_peered":"2024-03-14T21:14:03.028019+0000","last_unstale":"2024-03-14T22:06:09.178102+0000","last_undegraded":"2024-03-14T22:06:09.178102+0000","last_fullsized":"2024-03-14T22:06:09.178102+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10416","last_scrub_stamp":"2024-03-14T22:06:09.178081+0000","last_deep_scrub":"23'10416","last_deep_scrub_stamp":"2024-03-14T22:06:09.178081+0000","last_clean_scrub_stamp":"2024-03-14T22:06:09.178081+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:39.997111+0000","scrub_duration":0.014718489,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":8793,"num_write_kb":111405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'9704","reported_seq":9754,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:52.497668+0000","last_change":"2024-03-14T22:05:52.497829+0000","last_active":"2024-03-14T22:05:52.497668+0000","last_peered":"2024-03-14T22:05:52.497668+0000","last_clean":"2024-03-14T22:05:52.497668+0000","last_became_active":"2024-03-14T21:14:03.032874+0000","last_became_peered":"2024-03-14T21:14:03.032874+0000","last_unstale":"2024-03-14T22:05:52.497668+0000","last_undegraded":"2024-03-14T22:05:52.497668+0000","last_fullsized":"2024-03-14T22:05:52.497668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9704","last_scrub_stamp":"2024-03-14T22:05:52.497623+0000","last_deep_scrub":"23'9704","last_deep_scrub_stamp":"2024-03-14T22:05:52.497623+0000","last_clean_scrub_stamp":"2024-03-14T22:05:52.497623+0000","objects_scrubbed":0,"log_size":9704,"log_dups_size":0,"ondisk_log_size":9704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:42.520325+0000","scrub_duration":0.011931638,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8012,"num_write_kb":95820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'9661","reported_seq":9721,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.646033+0000","last_change":"2024-03-14T22:06:01.646136+0000","last_active":"2024-03-14T22:06:01.646033+0000","last_peered":"2024-03-14T22:06:01.646033+0000","last_clean":"2024-03-14T22:06:01.646033+0000","last_became_active":"2024-03-14T21:14:03.027452+0000","last_became_peered":"2024-03-14T21:14:03.027452+0000","last_unstale":"2024-03-14T22:06:01.646033+0000","last_undegraded":"2024-03-14T22:06:01.646033+0000","last_fullsized":"2024-03-14T22:06:01.646033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9661","last_scrub_stamp":"2024-03-14T22:06:01.646010+0000","last_deep_scrub":"23'9661","last_deep_scrub_stamp":"2024-03-14T22:06:01.646010+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.646010+0000","objects_scrubbed":0,"log_size":9661,"log_dups_size":0,"ondisk_log_size":9661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:19:19.857867+0000","scrub_duration":0.015379894999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3572,"num_write":8076,"num_write_kb":107857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'8662","reported_seq":8712,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:53.158066+0000","last_change":"2024-03-14T22:05:53.158066+0000","last_active":"2024-03-14T22:05:53.158066+0000","last_peered":"2024-03-14T22:05:53.158066+0000","last_clean":"2024-03-14T22:05:53.158066+0000","last_became_active":"2024-03-14T21:14:44.858906+0000","last_became_peered":"2024-03-14T21:14:44.858906+0000","last_unstale":"2024-03-14T22:05:53.158066+0000","last_undegraded":"2024-03-14T22:05:53.158066+0000","last_fullsized":"2024-03-14T22:05:53.158066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8662","last_scrub_stamp":"2024-03-14T22:05:53.157928+0000","last_deep_scrub":"23'8662","last_deep_scrub_stamp":"2024-03-14T22:05:53.157928+0000","last_clean_scrub_stamp":"2024-03-14T22:05:53.157928+0000","objects_scrubbed":0,"log_size":8662,"log_dups_size":0,"ondisk_log_size":8662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:52.457213+0000","scrub_duration":0.0057217580000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1528,"num_write":7020,"num_write_kb":102847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'10199","reported_seq":10247,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:46.076003+0000","last_change":"2024-03-14T22:05:46.076125+0000","last_active":"2024-03-14T22:05:46.076003+0000","last_peered":"2024-03-14T22:05:46.076003+0000","last_clean":"2024-03-14T22:05:46.076003+0000","last_became_active":"2024-03-14T21:14:03.029563+0000","last_became_peered":"2024-03-14T21:14:03.029563+0000","last_unstale":"2024-03-14T22:05:46.076003+0000","last_undegraded":"2024-03-14T22:05:46.076003+0000","last_fullsized":"2024-03-14T22:05:46.076003+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10199","last_scrub_stamp":"2024-03-14T22:05:46.075978+0000","last_deep_scrub":"23'10199","last_deep_scrub_stamp":"2024-03-14T22:05:46.075978+0000","last_clean_scrub_stamp":"2024-03-14T22:05:46.075978+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:28.104989+0000","scrub_duration":0.011461096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":510,"num_write":8552,"num_write_kb":105487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'10521","reported_seq":10568,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.528915+0000","last_change":"2024-03-14T22:06:01.528915+0000","last_active":"2024-03-14T22:06:01.528915+0000","last_peered":"2024-03-14T22:06:01.528915+0000","last_clean":"2024-03-14T22:06:01.528915+0000","last_became_active":"2024-03-14T21:14:03.033735+0000","last_became_peered":"2024-03-14T21:14:03.033735+0000","last_unstale":"2024-03-14T22:06:01.528915+0000","last_undegraded":"2024-03-14T22:06:01.528915+0000","last_fullsized":"2024-03-14T22:06:01.528915+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10521","last_scrub_stamp":"2024-03-14T22:06:01.528722+0000","last_deep_scrub":"23'10521","last_deep_scrub_stamp":"2024-03-14T22:06:01.528722+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.528722+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:43:24.576763+0000","scrub_duration":0.012349428000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":118219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'10900","reported_seq":10951,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:56.603066+0000","last_change":"2024-03-14T22:05:56.603174+0000","last_active":"2024-03-14T22:05:56.603066+0000","last_peered":"2024-03-14T22:05:56.603066+0000","last_clean":"2024-03-14T22:05:56.603066+0000","last_became_active":"2024-03-14T21:14:03.033049+0000","last_became_peered":"2024-03-14T21:14:03.033049+0000","last_unstale":"2024-03-14T22:05:56.603066+0000","last_undegraded":"2024-03-14T22:05:56.603066+0000","last_fullsized":"2024-03-14T22:05:56.603066+0000","mapping_epoch":19,"log_start":"23'800","ondisk_log_start":"23'800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10900","last_scrub_stamp":"2024-03-14T22:05:56.603044+0000","last_deep_scrub":"23'10900","last_deep_scrub_stamp":"2024-03-14T22:05:56.603044+0000","last_clean_scrub_stamp":"2024-03-14T22:05:56.603044+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:46.625746+0000","scrub_duration":0.013839676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1020,"num_write":9315,"num_write_kb":117297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'10962","reported_seq":11022,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.181353+0000","last_change":"2024-03-14T22:06:05.181507+0000","last_active":"2024-03-14T22:06:05.181353+0000","last_peered":"2024-03-14T22:06:05.181353+0000","last_clean":"2024-03-14T22:06:05.181353+0000","last_became_active":"2024-03-14T21:14:03.034806+0000","last_became_peered":"2024-03-14T21:14:03.034806+0000","last_unstale":"2024-03-14T22:06:05.181353+0000","last_undegraded":"2024-03-14T22:06:05.181353+0000","last_fullsized":"2024-03-14T22:06:05.181353+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10962","last_scrub_stamp":"2024-03-14T22:06:05.181330+0000","last_deep_scrub":"23'10962","last_deep_scrub_stamp":"2024-03-14T22:06:05.181330+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.181330+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:15.966373+0000","scrub_duration":0.011570298999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3050,"num_write":9338,"num_write_kb":109552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'10723","reported_seq":10772,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:08.662770+0000","last_change":"2024-03-14T22:06:08.662902+0000","last_active":"2024-03-14T22:06:08.662770+0000","last_peered":"2024-03-14T22:06:08.662770+0000","last_clean":"2024-03-14T22:06:08.662770+0000","last_became_active":"2024-03-14T21:14:03.031183+0000","last_became_peered":"2024-03-14T21:14:03.031183+0000","last_unstale":"2024-03-14T22:06:08.662770+0000","last_undegraded":"2024-03-14T22:06:08.662770+0000","last_fullsized":"2024-03-14T22:06:08.662770+0000","mapping_epoch":19,"log_start":"23'700","ondisk_log_start":"23'700","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10723","last_scrub_stamp":"2024-03-14T22:06:08.662746+0000","last_deep_scrub":"23'10723","last_deep_scrub_stamp":"2024-03-14T22:06:08.662746+0000","last_clean_scrub_stamp":"2024-03-14T22:06:08.662746+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:27:41.265598+0000","scrub_duration":0.011816327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":9066,"num_write_kb":118743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'10565","reported_seq":10614,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:03.942607+0000","last_change":"2024-03-14T22:06:03.942724+0000","last_active":"2024-03-14T22:06:03.942607+0000","last_peered":"2024-03-14T22:06:03.942607+0000","last_clean":"2024-03-14T22:06:03.942607+0000","last_became_active":"2024-03-14T21:14:03.041073+0000","last_became_peered":"2024-03-14T21:14:03.041073+0000","last_unstale":"2024-03-14T22:06:03.942607+0000","last_undegraded":"2024-03-14T22:06:03.942607+0000","last_fullsized":"2024-03-14T22:06:03.942607+0000","mapping_epoch":19,"log_start":"23'500","ondisk_log_start":"23'500","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10565","last_scrub_stamp":"2024-03-14T22:06:03.942584+0000","last_deep_scrub":"23'10565","last_deep_scrub_stamp":"2024-03-14T22:06:03.942584+0000","last_clean_scrub_stamp":"2024-03-14T22:06:03.942584+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:34.761614+0000","scrub_duration":0.013751275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8950,"num_write_kb":109690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'11180","reported_seq":11234,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:49.737865+0000","last_change":"2024-03-14T22:05:49.737865+0000","last_active":"2024-03-14T22:05:49.737865+0000","last_peered":"2024-03-14T22:05:49.737865+0000","last_clean":"2024-03-14T22:05:49.737865+0000","last_became_active":"2024-03-14T21:14:03.038267+0000","last_became_peered":"2024-03-14T21:14:03.038267+0000","last_unstale":"2024-03-14T22:05:49.737865+0000","last_undegraded":"2024-03-14T22:05:49.737865+0000","last_fullsized":"2024-03-14T22:05:49.737865+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11180","last_scrub_stamp":"2024-03-14T22:05:49.737563+0000","last_deep_scrub":"23'11180","last_deep_scrub_stamp":"2024-03-14T22:05:49.737563+0000","last_clean_scrub_stamp":"2024-03-14T22:05:49.737563+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:00.522606+0000","scrub_duration":0.013646538999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1530,"num_write":9571,"num_write_kb":127549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'10195","reported_seq":10245,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:50.983228+0000","last_change":"2024-03-14T22:05:50.983228+0000","last_active":"2024-03-14T22:05:50.983228+0000","last_peered":"2024-03-14T22:05:50.983228+0000","last_clean":"2024-03-14T22:05:50.983228+0000","last_became_active":"2024-03-14T21:14:03.042900+0000","last_became_peered":"2024-03-14T21:14:03.042900+0000","last_unstale":"2024-03-14T22:05:50.983228+0000","last_undegraded":"2024-03-14T22:05:50.983228+0000","last_fullsized":"2024-03-14T22:05:50.983228+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10195","last_scrub_stamp":"2024-03-14T22:05:50.983096+0000","last_deep_scrub":"23'10195","last_deep_scrub_stamp":"2024-03-14T22:05:50.983096+0000","last_clean_scrub_stamp":"2024-03-14T22:05:50.983096+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:41.005798+0000","scrub_duration":0.0067038030000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":512,"num_write":8582,"num_write_kb":110246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'12334","reported_seq":12388,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.612993+0000","last_change":"2024-03-14T22:06:05.613124+0000","last_active":"2024-03-14T22:06:05.612993+0000","last_peered":"2024-03-14T22:06:05.612993+0000","last_clean":"2024-03-14T22:06:05.612993+0000","last_became_active":"2024-03-14T21:14:03.038792+0000","last_became_peered":"2024-03-14T21:14:03.038792+0000","last_unstale":"2024-03-14T22:06:05.612993+0000","last_undegraded":"2024-03-14T22:06:05.612993+0000","last_fullsized":"2024-03-14T22:06:05.612993+0000","mapping_epoch":19,"log_start":"23'2300","ondisk_log_start":"23'2300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12334","last_scrub_stamp":"2024-03-14T22:06:05.612968+0000","last_deep_scrub":"23'12334","last_deep_scrub_stamp":"2024-03-14T22:06:05.612968+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.612968+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:16.398011+0000","scrub_duration":0.010715311,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10697,"num_write_kb":121303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'9243","reported_seq":9299,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.500698+0000","last_change":"2024-03-14T22:06:01.500698+0000","last_active":"2024-03-14T22:06:01.500698+0000","last_peered":"2024-03-14T22:06:01.500698+0000","last_clean":"2024-03-14T22:06:01.500698+0000","last_became_active":"2024-03-14T21:14:03.031496+0000","last_became_peered":"2024-03-14T21:14:03.031496+0000","last_unstale":"2024-03-14T22:06:01.500698+0000","last_undegraded":"2024-03-14T22:06:01.500698+0000","last_fullsized":"2024-03-14T22:06:01.500698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9243","last_scrub_stamp":"2024-03-14T22:06:01.500545+0000","last_deep_scrub":"23'9243","last_deep_scrub_stamp":"2024-03-14T22:06:01.500545+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.500545+0000","objects_scrubbed":0,"log_size":9243,"log_dups_size":0,"ondisk_log_size":9243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:12.285588+0000","scrub_duration":0.0091242010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":7601,"num_write_kb":100407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'10312","reported_seq":10360,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:49.938608+0000","last_change":"2024-03-14T22:05:49.938608+0000","last_active":"2024-03-14T22:05:49.938608+0000","last_peered":"2024-03-14T22:05:49.938608+0000","last_clean":"2024-03-14T22:05:49.938608+0000","last_became_active":"2024-03-14T21:14:03.040450+0000","last_became_peered":"2024-03-14T21:14:03.040450+0000","last_unstale":"2024-03-14T22:05:49.938608+0000","last_undegraded":"2024-03-14T22:05:49.938608+0000","last_fullsized":"2024-03-14T22:05:49.938608+0000","mapping_epoch":19,"log_start":"23'300","ondisk_log_start":"23'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10312","last_scrub_stamp":"2024-03-14T22:05:49.938442+0000","last_deep_scrub":"23'10312","last_deep_scrub_stamp":"2024-03-14T22:05:49.938442+0000","last_clean_scrub_stamp":"2024-03-14T22:05:49.938442+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:20.664950+0000","scrub_duration":0.012238142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":108137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'11957","reported_seq":12018,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:01.174827+0000","last_change":"2024-03-14T22:06:01.174827+0000","last_active":"2024-03-14T22:06:01.174827+0000","last_peered":"2024-03-14T22:06:01.174827+0000","last_clean":"2024-03-14T22:06:01.174827+0000","last_became_active":"2024-03-14T21:14:44.746271+0000","last_became_peered":"2024-03-14T21:14:44.746271+0000","last_unstale":"2024-03-14T22:06:01.174827+0000","last_undegraded":"2024-03-14T22:06:01.174827+0000","last_fullsized":"2024-03-14T22:06:01.174827+0000","mapping_epoch":22,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11957","last_scrub_stamp":"2024-03-14T22:06:01.174676+0000","last_deep_scrub":"23'11957","last_deep_scrub_stamp":"2024-03-14T22:06:01.174676+0000","last_clean_scrub_stamp":"2024-03-14T22:06:01.174676+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:28:00.473961+0000","scrub_duration":0.0096213229999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":10306,"num_write_kb":120667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'11954","reported_seq":12009,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.635340+0000","last_change":"2024-03-14T22:06:05.635340+0000","last_active":"2024-03-14T22:06:05.635340+0000","last_peered":"2024-03-14T22:06:05.635340+0000","last_clean":"2024-03-14T22:06:05.635340+0000","last_became_active":"2024-03-14T21:14:03.038274+0000","last_became_peered":"2024-03-14T21:14:03.038274+0000","last_unstale":"2024-03-14T22:06:05.635340+0000","last_undegraded":"2024-03-14T22:06:05.635340+0000","last_fullsized":"2024-03-14T22:06:05.635340+0000","mapping_epoch":19,"log_start":"23'1900","ondisk_log_start":"23'1900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11954","last_scrub_stamp":"2024-03-14T22:06:05.635217+0000","last_deep_scrub":"23'11954","last_deep_scrub_stamp":"2024-03-14T22:06:05.635217+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.635217+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:06:57.722318+0000","scrub_duration":0.012719486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":10416,"num_write_kb":126678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'11296","reported_seq":11356,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.103982+0000","last_change":"2024-03-14T22:05:47.103982+0000","last_active":"2024-03-14T22:05:47.103982+0000","last_peered":"2024-03-14T22:05:47.103982+0000","last_clean":"2024-03-14T22:05:47.103982+0000","last_became_active":"2024-03-14T21:14:03.029597+0000","last_became_peered":"2024-03-14T21:14:03.029597+0000","last_unstale":"2024-03-14T22:05:47.103982+0000","last_undegraded":"2024-03-14T22:05:47.103982+0000","last_fullsized":"2024-03-14T22:05:47.103982+0000","mapping_epoch":19,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11296","last_scrub_stamp":"2024-03-14T22:05:47.103819+0000","last_deep_scrub":"23'11296","last_deep_scrub_stamp":"2024-03-14T22:05:47.103819+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.103819+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:17.830327+0000","scrub_duration":0.013053291999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2806,"num_write":9622,"num_write_kb":118224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'8637","reported_seq":8687,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:06.656145+0000","last_change":"2024-03-14T22:06:06.656145+0000","last_active":"2024-03-14T22:06:06.656145+0000","last_peered":"2024-03-14T22:06:06.656145+0000","last_clean":"2024-03-14T22:06:06.656145+0000","last_became_active":"2024-03-14T21:14:03.033871+0000","last_became_peered":"2024-03-14T21:14:03.033871+0000","last_unstale":"2024-03-14T22:06:06.656145+0000","last_undegraded":"2024-03-14T22:06:06.656145+0000","last_fullsized":"2024-03-14T22:06:06.656145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8637","last_scrub_stamp":"2024-03-14T22:06:06.655961+0000","last_deep_scrub":"23'8637","last_deep_scrub_stamp":"2024-03-14T22:06:06.655961+0000","last_clean_scrub_stamp":"2024-03-14T22:06:06.655961+0000","objects_scrubbed":0,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:56:00.034958+0000","scrub_duration":0.013905326000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":7051,"num_write_kb":105885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'9053","reported_seq":9101,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.818239+0000","last_change":"2024-03-14T22:05:37.818351+0000","last_active":"2024-03-14T22:05:37.818239+0000","last_peered":"2024-03-14T22:05:37.818239+0000","last_clean":"2024-03-14T22:05:37.818239+0000","last_became_active":"2024-03-14T21:14:03.037642+0000","last_became_peered":"2024-03-14T21:14:03.037642+0000","last_unstale":"2024-03-14T22:05:37.818239+0000","last_undegraded":"2024-03-14T22:05:37.818239+0000","last_fullsized":"2024-03-14T22:05:37.818239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9053","last_scrub_stamp":"2024-03-14T22:05:37.818216+0000","last_deep_scrub":"23'9053","last_deep_scrub_stamp":"2024-03-14T22:05:37.818216+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.818216+0000","objects_scrubbed":0,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:12:07.513721+0000","scrub_duration":0.014777571999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":7364,"num_write_kb":104642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'10865","reported_seq":10910,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.106755+0000","last_change":"2024-03-14T22:05:37.106859+0000","last_active":"2024-03-14T22:05:37.106755+0000","last_peered":"2024-03-14T22:05:37.106755+0000","last_clean":"2024-03-14T22:05:37.106755+0000","last_became_active":"2024-03-14T21:14:02.655466+0000","last_became_peered":"2024-03-14T21:14:02.655466+0000","last_unstale":"2024-03-14T22:05:37.106755+0000","last_undegraded":"2024-03-14T22:05:37.106755+0000","last_fullsized":"2024-03-14T22:05:37.106755+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10865","last_scrub_stamp":"2024-03-14T22:05:37.106731+0000","last_deep_scrub":"23'10865","last_deep_scrub_stamp":"2024-03-14T22:05:37.106731+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.106731+0000","objects_scrubbed":4,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:08.649923+0000","scrub_duration":0.054954579000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10855,"num_write_kb":88672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'9539","reported_seq":9590,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.102174+0000","last_change":"2024-03-14T22:05:43.102310+0000","last_active":"2024-03-14T22:05:43.102174+0000","last_peered":"2024-03-14T22:05:43.102174+0000","last_clean":"2024-03-14T22:05:43.102174+0000","last_became_active":"2024-03-14T21:14:03.032718+0000","last_became_peered":"2024-03-14T21:14:03.032718+0000","last_unstale":"2024-03-14T22:05:43.102174+0000","last_undegraded":"2024-03-14T22:05:43.102174+0000","last_fullsized":"2024-03-14T22:05:43.102174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9539","last_scrub_stamp":"2024-03-14T22:05:43.102152+0000","last_deep_scrub":"23'9539","last_deep_scrub_stamp":"2024-03-14T22:05:43.102152+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.102152+0000","objects_scrubbed":0,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:27.735439+0000","scrub_duration":0.010746225999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":7923,"num_write_kb":107722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10915","reported_seq":10960,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.455431+0000","last_change":"2024-03-14T22:05:43.455532+0000","last_active":"2024-03-14T22:05:43.455431+0000","last_peered":"2024-03-14T22:05:43.455431+0000","last_clean":"2024-03-14T22:05:43.455431+0000","last_became_active":"2024-03-14T21:14:02.662639+0000","last_became_peered":"2024-03-14T21:14:02.662639+0000","last_unstale":"2024-03-14T22:05:43.455431+0000","last_undegraded":"2024-03-14T22:05:43.455431+0000","last_fullsized":"2024-03-14T22:05:43.455431+0000","mapping_epoch":18,"log_start":"23'900","ondisk_log_start":"23'900","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10915","last_scrub_stamp":"2024-03-14T22:05:43.455407+0000","last_deep_scrub":"23'10915","last_deep_scrub_stamp":"2024-03-14T22:05:43.455407+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.455407+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:28.088694+0000","scrub_duration":0.005473159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":10937,"num_write_kb":77181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'7037","reported_seq":7083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.042078+0000","last_change":"2024-03-14T22:05:38.042187+0000","last_active":"2024-03-14T22:05:38.042078+0000","last_peered":"2024-03-14T22:05:38.042078+0000","last_clean":"2024-03-14T22:05:38.042078+0000","last_became_active":"2024-03-14T21:14:02.660306+0000","last_became_peered":"2024-03-14T21:14:02.660306+0000","last_unstale":"2024-03-14T22:05:38.042078+0000","last_undegraded":"2024-03-14T22:05:38.042078+0000","last_fullsized":"2024-03-14T22:05:38.042078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7037","last_scrub_stamp":"2024-03-14T22:05:38.042054+0000","last_deep_scrub":"23'7037","last_deep_scrub_stamp":"2024-03-14T22:05:38.042054+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.042054+0000","objects_scrubbed":1,"log_size":7037,"log_dups_size":0,"ondisk_log_size":7037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:22.675341+0000","scrub_duration":0.019933317999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7046,"num_write_kb":58697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'11487","reported_seq":11533,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:00.969365+0000","last_change":"2024-03-14T22:06:00.969520+0000","last_active":"2024-03-14T22:06:00.969365+0000","last_peered":"2024-03-14T22:06:00.969365+0000","last_clean":"2024-03-14T22:06:00.969365+0000","last_became_active":"2024-03-14T21:14:03.037131+0000","last_became_peered":"2024-03-14T21:14:03.037131+0000","last_unstale":"2024-03-14T22:06:00.969365+0000","last_undegraded":"2024-03-14T22:06:00.969365+0000","last_fullsized":"2024-03-14T22:06:00.969365+0000","mapping_epoch":19,"log_start":"23'1400","ondisk_log_start":"23'1400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11487","last_scrub_stamp":"2024-03-14T22:06:00.969339+0000","last_deep_scrub":"23'11487","last_deep_scrub_stamp":"2024-03-14T22:06:00.969339+0000","last_clean_scrub_stamp":"2024-03-14T22:06:00.969339+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:54.348337+0000","scrub_duration":0.015493251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":9890,"num_write_kb":106254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'11077","reported_seq":11123,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:40.604311+0000","last_change":"2024-03-14T22:05:40.604410+0000","last_active":"2024-03-14T22:05:40.604311+0000","last_peered":"2024-03-14T22:05:40.604311+0000","last_clean":"2024-03-14T22:05:40.604311+0000","last_became_active":"2024-03-14T21:14:02.660976+0000","last_became_peered":"2024-03-14T21:14:02.660976+0000","last_unstale":"2024-03-14T22:05:40.604311+0000","last_undegraded":"2024-03-14T22:05:40.604311+0000","last_fullsized":"2024-03-14T22:05:40.604311+0000","mapping_epoch":18,"log_start":"23'1000","ondisk_log_start":"23'1000","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11077","last_scrub_stamp":"2024-03-14T22:05:40.604286+0000","last_deep_scrub":"23'11077","last_deep_scrub_stamp":"2024-03-14T22:05:40.604286+0000","last_clean_scrub_stamp":"2024-03-14T22:05:40.604286+0000","objects_scrubbed":1,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:30.626988+0000","scrub_duration":0.021895357000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":11078,"num_write_kb":81064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'10475","reported_seq":10531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:05.394404+0000","last_change":"2024-03-14T22:06:05.394533+0000","last_active":"2024-03-14T22:06:05.394404+0000","last_peered":"2024-03-14T22:06:05.394404+0000","last_clean":"2024-03-14T22:06:05.394404+0000","last_became_active":"2024-03-14T21:14:03.035397+0000","last_became_peered":"2024-03-14T21:14:03.035397+0000","last_unstale":"2024-03-14T22:06:05.394404+0000","last_undegraded":"2024-03-14T22:06:05.394404+0000","last_fullsized":"2024-03-14T22:06:05.394404+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10475","last_scrub_stamp":"2024-03-14T22:06:05.394380+0000","last_deep_scrub":"23'10475","last_deep_scrub_stamp":"2024-03-14T22:06:05.394380+0000","last_clean_scrub_stamp":"2024-03-14T22:06:05.394380+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:36.213410+0000","scrub_duration":0.0089458139999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2800,"num_write":8909,"num_write_kb":114509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'8287","reported_seq":8938,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:54.608435+0000","last_change":"2024-03-14T22:05:54.608544+0000","last_active":"2024-03-14T22:05:54.608435+0000","last_peered":"2024-03-14T22:05:54.608435+0000","last_clean":"2024-03-14T22:05:54.608435+0000","last_became_active":"2024-03-14T21:14:02.662619+0000","last_became_peered":"2024-03-14T21:14:02.662619+0000","last_unstale":"2024-03-14T22:05:54.608435+0000","last_undegraded":"2024-03-14T22:05:54.608435+0000","last_fullsized":"2024-03-14T22:05:54.608435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8287","last_scrub_stamp":"2024-03-14T22:05:54.608410+0000","last_deep_scrub":"23'8287","last_deep_scrub_stamp":"2024-03-14T22:05:54.608410+0000","last_clean_scrub_stamp":"2024-03-14T22:05:54.608410+0000","objects_scrubbed":1,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:36.637420+0000","scrub_duration":0.027372786,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":607,"num_read_kb":4494,"num_write":8292,"num_write_kb":66794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'10203","reported_seq":10253,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:53.622084+0000","last_change":"2024-03-14T22:05:53.622208+0000","last_active":"2024-03-14T22:05:53.622084+0000","last_peered":"2024-03-14T22:05:53.622084+0000","last_clean":"2024-03-14T22:05:53.622084+0000","last_became_active":"2024-03-14T21:14:03.031115+0000","last_became_peered":"2024-03-14T21:14:03.031115+0000","last_unstale":"2024-03-14T22:05:53.622084+0000","last_undegraded":"2024-03-14T22:05:53.622084+0000","last_fullsized":"2024-03-14T22:05:53.622084+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10203","last_scrub_stamp":"2024-03-14T22:05:53.622058+0000","last_deep_scrub":"23'10203","last_deep_scrub_stamp":"2024-03-14T22:05:53.622058+0000","last_clean_scrub_stamp":"2024-03-14T22:05:53.622058+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:45:24.441088+0000","scrub_duration":0.015986779,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8609,"num_write_kb":104201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'10695","reported_seq":10741,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.657443+0000","last_change":"2024-03-14T22:05:39.657538+0000","last_active":"2024-03-14T22:05:39.657443+0000","last_peered":"2024-03-14T22:05:39.657443+0000","last_clean":"2024-03-14T22:05:39.657443+0000","last_became_active":"2024-03-14T21:14:02.660092+0000","last_became_peered":"2024-03-14T21:14:02.660092+0000","last_unstale":"2024-03-14T22:05:39.657443+0000","last_undegraded":"2024-03-14T22:05:39.657443+0000","last_fullsized":"2024-03-14T22:05:39.657443+0000","mapping_epoch":18,"log_start":"23'600","ondisk_log_start":"23'600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10695","last_scrub_stamp":"2024-03-14T22:05:39.657419+0000","last_deep_scrub":"23'10695","last_deep_scrub_stamp":"2024-03-14T22:05:39.657419+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.657419+0000","objects_scrubbed":6,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:10.383927+0000","scrub_duration":0.080830166999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":92935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'9099","reported_seq":9148,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:53.162311+0000","last_change":"2024-03-14T22:05:53.162394+0000","last_active":"2024-03-14T22:05:53.162311+0000","last_peered":"2024-03-14T22:05:53.162311+0000","last_clean":"2024-03-14T22:05:53.162311+0000","last_became_active":"2024-03-14T21:14:03.030511+0000","last_became_peered":"2024-03-14T21:14:03.030511+0000","last_unstale":"2024-03-14T22:05:53.162311+0000","last_undegraded":"2024-03-14T22:05:53.162311+0000","last_fullsized":"2024-03-14T22:05:53.162311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9099","last_scrub_stamp":"2024-03-14T22:05:53.162292+0000","last_deep_scrub":"23'9099","last_deep_scrub_stamp":"2024-03-14T22:05:53.162292+0000","last_clean_scrub_stamp":"2024-03-14T22:05:53.162292+0000","objects_scrubbed":0,"log_size":9099,"log_dups_size":0,"ondisk_log_size":9099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:35.191303+0000","scrub_duration":0.0086207539999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7455,"num_write_kb":104328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9150","reported_seq":9196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:51.453255+0000","last_change":"2024-03-14T22:05:51.453370+0000","last_active":"2024-03-14T22:05:51.453255+0000","last_peered":"2024-03-14T22:05:51.453255+0000","last_clean":"2024-03-14T22:05:51.453255+0000","last_became_active":"2024-03-14T21:14:02.653917+0000","last_became_peered":"2024-03-14T21:14:02.653917+0000","last_unstale":"2024-03-14T22:05:51.453255+0000","last_undegraded":"2024-03-14T22:05:51.453255+0000","last_fullsized":"2024-03-14T22:05:51.453255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9150","last_scrub_stamp":"2024-03-14T22:05:51.453231+0000","last_deep_scrub":"23'9150","last_deep_scrub_stamp":"2024-03-14T22:05:51.453231+0000","last_clean_scrub_stamp":"2024-03-14T22:05:51.453231+0000","objects_scrubbed":2,"log_size":9150,"log_dups_size":0,"ondisk_log_size":9150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:22.179739+0000","scrub_duration":0.01173714,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9159,"num_write_kb":60302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'11329","reported_seq":11385,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.792145+0000","last_change":"2024-03-14T22:05:34.792145+0000","last_active":"2024-03-14T22:05:34.792145+0000","last_peered":"2024-03-14T22:05:34.792145+0000","last_clean":"2024-03-14T22:05:34.792145+0000","last_became_active":"2024-03-14T21:14:03.032235+0000","last_became_peered":"2024-03-14T21:14:03.032235+0000","last_unstale":"2024-03-14T22:05:34.792145+0000","last_undegraded":"2024-03-14T22:05:34.792145+0000","last_fullsized":"2024-03-14T22:05:34.792145+0000","mapping_epoch":19,"log_start":"23'1300","ondisk_log_start":"23'1300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11329","last_scrub_stamp":"2024-03-14T22:05:34.791977+0000","last_deep_scrub":"23'11329","last_deep_scrub_stamp":"2024-03-14T22:05:34.791977+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.791977+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:36:18.331402+0000","scrub_duration":0.011767835000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2542,"num_write":9699,"num_write_kb":113170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'13354","reported_seq":13955,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.451109+0000","last_change":"2024-03-14T22:05:38.451212+0000","last_active":"2024-03-14T22:05:38.451109+0000","last_peered":"2024-03-14T22:05:38.451109+0000","last_clean":"2024-03-14T22:05:38.451109+0000","last_became_active":"2024-03-14T21:14:02.658573+0000","last_became_peered":"2024-03-14T21:14:02.658573+0000","last_unstale":"2024-03-14T22:05:38.451109+0000","last_undegraded":"2024-03-14T22:05:38.451109+0000","last_fullsized":"2024-03-14T22:05:38.451109+0000","mapping_epoch":18,"log_start":"23'3300","ondisk_log_start":"23'3300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13354","last_scrub_stamp":"2024-03-14T22:05:38.451087+0000","last_deep_scrub":"23'13354","last_deep_scrub_stamp":"2024-03-14T22:05:38.451087+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.451087+0000","objects_scrubbed":4,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:50.713752+0000","scrub_duration":0.026780381999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":557,"num_read_kb":4455,"num_write":13349,"num_write_kb":107014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'9106","reported_seq":9164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:13.519887+0000","last_change":"2024-03-14T22:06:13.520019+0000","last_active":"2024-03-14T22:06:13.519887+0000","last_peered":"2024-03-14T22:06:13.519887+0000","last_clean":"2024-03-14T22:06:13.519887+0000","last_became_active":"2024-03-14T21:14:03.031927+0000","last_became_peered":"2024-03-14T21:14:03.031927+0000","last_unstale":"2024-03-14T22:06:13.519887+0000","last_undegraded":"2024-03-14T22:06:13.519887+0000","last_fullsized":"2024-03-14T22:06:13.519887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9106","last_scrub_stamp":"2024-03-14T22:06:13.519864+0000","last_deep_scrub":"23'9106","last_deep_scrub_stamp":"2024-03-14T22:06:13.519864+0000","last_clean_scrub_stamp":"2024-03-14T22:06:13.519864+0000","objects_scrubbed":0,"log_size":9106,"log_dups_size":0,"ondisk_log_size":9106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:07:05.606965+0000","scrub_duration":0.011786359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3318,"num_write":7536,"num_write_kb":95248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'8011","reported_seq":8058,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:37.179662+0000","last_change":"2024-03-14T22:05:37.179778+0000","last_active":"2024-03-14T22:05:37.179662+0000","last_peered":"2024-03-14T22:05:37.179662+0000","last_clean":"2024-03-14T22:05:37.179662+0000","last_became_active":"2024-03-14T21:14:02.662806+0000","last_became_peered":"2024-03-14T21:14:02.662806+0000","last_unstale":"2024-03-14T22:05:37.179662+0000","last_undegraded":"2024-03-14T22:05:37.179662+0000","last_fullsized":"2024-03-14T22:05:37.179662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8011","last_scrub_stamp":"2024-03-14T22:05:37.179641+0000","last_deep_scrub":"23'8011","last_deep_scrub_stamp":"2024-03-14T22:05:37.179641+0000","last_clean_scrub_stamp":"2024-03-14T22:05:37.179641+0000","objects_scrubbed":2,"log_size":8011,"log_dups_size":0,"ondisk_log_size":8011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:49.442306+0000","scrub_duration":0.011524995999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8008,"num_write_kb":62700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'9649","reported_seq":9700,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:54.202635+0000","last_change":"2024-03-14T22:05:54.202762+0000","last_active":"2024-03-14T22:05:54.202635+0000","last_peered":"2024-03-14T22:05:54.202635+0000","last_clean":"2024-03-14T22:05:54.202635+0000","last_became_active":"2024-03-14T21:14:03.033800+0000","last_became_peered":"2024-03-14T21:14:03.033800+0000","last_unstale":"2024-03-14T22:05:54.202635+0000","last_undegraded":"2024-03-14T22:05:54.202635+0000","last_fullsized":"2024-03-14T22:05:54.202635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9649","last_scrub_stamp":"2024-03-14T22:05:54.202612+0000","last_deep_scrub":"23'9649","last_deep_scrub_stamp":"2024-03-14T22:05:54.202612+0000","last_clean_scrub_stamp":"2024-03-14T22:05:54.202612+0000","objects_scrubbed":0,"log_size":9649,"log_dups_size":0,"ondisk_log_size":9649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:24.929120+0000","scrub_duration":0.013083743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8033,"num_write_kb":100651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5758","reported_seq":5805,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:45.712184+0000","last_change":"2024-03-14T22:05:45.712286+0000","last_active":"2024-03-14T22:05:45.712184+0000","last_peered":"2024-03-14T22:05:45.712184+0000","last_clean":"2024-03-14T22:05:45.712184+0000","last_became_active":"2024-03-14T21:14:02.659469+0000","last_became_peered":"2024-03-14T21:14:02.659469+0000","last_unstale":"2024-03-14T22:05:45.712184+0000","last_undegraded":"2024-03-14T22:05:45.712184+0000","last_fullsized":"2024-03-14T22:05:45.712184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5758","last_scrub_stamp":"2024-03-14T22:05:45.712161+0000","last_deep_scrub":"23'5758","last_deep_scrub_stamp":"2024-03-14T22:05:45.712161+0000","last_clean_scrub_stamp":"2024-03-14T22:05:45.712161+0000","objects_scrubbed":2,"log_size":5758,"log_dups_size":0,"ondisk_log_size":5758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:45.011446+0000","scrub_duration":0.027383683999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5754,"num_write_kb":48600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'9764","reported_seq":9812,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:02.491868+0000","last_change":"2024-03-14T22:06:02.492011+0000","last_active":"2024-03-14T22:06:02.491868+0000","last_peered":"2024-03-14T22:06:02.491868+0000","last_clean":"2024-03-14T22:06:02.491868+0000","last_became_active":"2024-03-14T21:14:03.032367+0000","last_became_peered":"2024-03-14T21:14:03.032367+0000","last_unstale":"2024-03-14T22:06:02.491868+0000","last_undegraded":"2024-03-14T22:06:02.491868+0000","last_fullsized":"2024-03-14T22:06:02.491868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9764","last_scrub_stamp":"2024-03-14T22:06:02.491844+0000","last_deep_scrub":"23'9764","last_deep_scrub_stamp":"2024-03-14T22:06:02.491844+0000","last_clean_scrub_stamp":"2024-03-14T22:06:02.491844+0000","objects_scrubbed":0,"log_size":9764,"log_dups_size":0,"ondisk_log_size":9764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:27:54.242135+0000","scrub_duration":0.011674722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8135,"num_write_kb":106503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5852","reported_seq":5899,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.442569+0000","last_change":"2024-03-14T22:05:42.442683+0000","last_active":"2024-03-14T22:05:42.442569+0000","last_peered":"2024-03-14T22:05:42.442569+0000","last_clean":"2024-03-14T22:05:42.442569+0000","last_became_active":"2024-03-14T21:14:02.663318+0000","last_became_peered":"2024-03-14T21:14:02.663318+0000","last_unstale":"2024-03-14T22:05:42.442569+0000","last_undegraded":"2024-03-14T22:05:42.442569+0000","last_fullsized":"2024-03-14T22:05:42.442569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5852","last_scrub_stamp":"2024-03-14T22:05:42.442545+0000","last_deep_scrub":"23'5852","last_deep_scrub_stamp":"2024-03-14T22:05:42.442545+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.442545+0000","objects_scrubbed":1,"log_size":5852,"log_dups_size":0,"ondisk_log_size":5852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:13.985738+0000","scrub_duration":0.016901003000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":52920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'9086","reported_seq":9142,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:55.633873+0000","last_change":"2024-03-14T22:05:55.634080+0000","last_active":"2024-03-14T22:05:55.633873+0000","last_peered":"2024-03-14T22:05:55.633873+0000","last_clean":"2024-03-14T22:05:55.633873+0000","last_became_active":"2024-03-14T21:14:03.033095+0000","last_became_peered":"2024-03-14T21:14:03.033095+0000","last_unstale":"2024-03-14T22:05:55.633873+0000","last_undegraded":"2024-03-14T22:05:55.633873+0000","last_fullsized":"2024-03-14T22:05:55.633873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9086","last_scrub_stamp":"2024-03-14T22:05:55.633836+0000","last_deep_scrub":"23'9086","last_deep_scrub_stamp":"2024-03-14T22:05:55.633836+0000","last_clean_scrub_stamp":"2024-03-14T22:05:55.633836+0000","objects_scrubbed":0,"log_size":9086,"log_dups_size":0,"ondisk_log_size":9086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:26.360344+0000","scrub_duration":0.014642007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2548,"num_write":7495,"num_write_kb":101746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8334","reported_seq":8381,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:30.179949+0000","last_change":"2024-03-14T22:05:30.180079+0000","last_active":"2024-03-14T22:05:30.179949+0000","last_peered":"2024-03-14T22:05:30.179949+0000","last_clean":"2024-03-14T22:05:30.179949+0000","last_became_active":"2024-03-14T21:14:02.654591+0000","last_became_peered":"2024-03-14T21:14:02.654591+0000","last_unstale":"2024-03-14T22:05:30.179949+0000","last_undegraded":"2024-03-14T22:05:30.179949+0000","last_fullsized":"2024-03-14T22:05:30.179949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8334","last_scrub_stamp":"2024-03-14T22:05:30.179928+0000","last_deep_scrub":"23'8334","last_deep_scrub_stamp":"2024-03-14T22:05:30.179928+0000","last_clean_scrub_stamp":"2024-03-14T22:05:30.179928+0000","objects_scrubbed":8,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:42.442593+0000","scrub_duration":0.057801277999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16724139,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":0,"num_write":8340,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'10136","reported_seq":10184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:06.386397+0000","last_change":"2024-03-14T22:06:06.386528+0000","last_active":"2024-03-14T22:06:06.386397+0000","last_peered":"2024-03-14T22:06:06.386397+0000","last_clean":"2024-03-14T22:06:06.386397+0000","last_became_active":"2024-03-14T21:14:44.859185+0000","last_became_peered":"2024-03-14T21:14:44.859185+0000","last_unstale":"2024-03-14T22:06:06.386397+0000","last_undegraded":"2024-03-14T22:06:06.386397+0000","last_fullsized":"2024-03-14T22:06:06.386397+0000","mapping_epoch":22,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10136","last_scrub_stamp":"2024-03-14T22:06:06.386374+0000","last_deep_scrub":"23'10136","last_deep_scrub_stamp":"2024-03-14T22:06:06.386374+0000","last_clean_scrub_stamp":"2024-03-14T22:06:06.386374+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:23:26.419647+0000","scrub_duration":0.013047168,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1022,"num_write":8491,"num_write_kb":113136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"23'8049","reported_seq":8097,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.782835+0000","last_change":"2024-03-14T22:05:31.782835+0000","last_active":"2024-03-14T22:05:31.782835+0000","last_peered":"2024-03-14T22:05:31.782835+0000","last_clean":"2024-03-14T22:05:31.782835+0000","last_became_active":"2024-03-14T21:14:02.663379+0000","last_became_peered":"2024-03-14T21:14:02.663379+0000","last_unstale":"2024-03-14T22:05:31.782835+0000","last_undegraded":"2024-03-14T22:05:31.782835+0000","last_fullsized":"2024-03-14T22:05:31.782835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8049","last_scrub_stamp":"2024-03-14T22:05:31.782651+0000","last_deep_scrub":"23'8049","last_deep_scrub_stamp":"2024-03-14T22:05:31.782651+0000","last_clean_scrub_stamp":"2024-03-14T22:05:31.782651+0000","objects_scrubbed":3,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T06:57:34.557091+0000","scrub_duration":0.036800939999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8051,"num_write_kb":62569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'9555","reported_seq":9610,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.307278+0000","last_change":"2024-03-14T22:05:34.307278+0000","last_active":"2024-03-14T22:05:34.307278+0000","last_peered":"2024-03-14T22:05:34.307278+0000","last_clean":"2024-03-14T22:05:34.307278+0000","last_became_active":"2024-03-14T21:14:03.030599+0000","last_became_peered":"2024-03-14T21:14:03.030599+0000","last_unstale":"2024-03-14T22:05:34.307278+0000","last_undegraded":"2024-03-14T22:05:34.307278+0000","last_fullsized":"2024-03-14T22:05:34.307278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9555","last_scrub_stamp":"2024-03-14T22:05:34.307098+0000","last_deep_scrub":"23'9555","last_deep_scrub_stamp":"2024-03-14T22:05:34.307098+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.307098+0000","objects_scrubbed":0,"log_size":9555,"log_dups_size":0,"ondisk_log_size":9555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:18.940385+0000","scrub_duration":0.012363498000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2042,"num_write":7926,"num_write_kb":109666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'10891","reported_seq":10939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.205897+0000","last_change":"2024-03-14T22:05:39.205897+0000","last_active":"2024-03-14T22:05:39.205897+0000","last_peered":"2024-03-14T22:05:39.205897+0000","last_clean":"2024-03-14T22:05:39.205897+0000","last_became_active":"2024-03-14T21:14:02.658824+0000","last_became_peered":"2024-03-14T21:14:02.658824+0000","last_unstale":"2024-03-14T22:05:39.205897+0000","last_undegraded":"2024-03-14T22:05:39.205897+0000","last_fullsized":"2024-03-14T22:05:39.205897+0000","mapping_epoch":18,"log_start":"23'800","ondisk_log_start":"23'800","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10891","last_scrub_stamp":"2024-03-14T22:05:39.205752+0000","last_deep_scrub":"23'10891","last_deep_scrub_stamp":"2024-03-14T22:05:39.205752+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.205752+0000","objects_scrubbed":2,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:13.412709+0000","scrub_duration":0.023607316999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10896,"num_write_kb":88926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'9015","reported_seq":9067,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:16.532555+0000","last_change":"2024-03-14T22:06:16.532679+0000","last_active":"2024-03-14T22:06:16.532555+0000","last_peered":"2024-03-14T22:06:16.532555+0000","last_clean":"2024-03-14T22:06:16.532555+0000","last_became_active":"2024-03-14T21:14:03.037546+0000","last_became_peered":"2024-03-14T21:14:03.037546+0000","last_unstale":"2024-03-14T22:06:16.532555+0000","last_undegraded":"2024-03-14T22:06:16.532555+0000","last_fullsized":"2024-03-14T22:06:16.532555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9015","last_scrub_stamp":"2024-03-14T22:06:16.532530+0000","last_deep_scrub":"23'9015","last_deep_scrub_stamp":"2024-03-14T22:06:16.532530+0000","last_clean_scrub_stamp":"2024-03-14T22:06:16.532530+0000","objects_scrubbed":0,"log_size":9015,"log_dups_size":0,"ondisk_log_size":9015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:27:49.135381+0000","scrub_duration":0.0084480260000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":7443,"num_write_kb":93234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'11502","reported_seq":11550,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:39.317932+0000","last_change":"2024-03-14T22:05:39.317932+0000","last_active":"2024-03-14T22:05:39.317932+0000","last_peered":"2024-03-14T22:05:39.317932+0000","last_clean":"2024-03-14T22:05:39.317932+0000","last_became_active":"2024-03-14T21:14:02.663087+0000","last_became_peered":"2024-03-14T21:14:02.663087+0000","last_unstale":"2024-03-14T22:05:39.317932+0000","last_undegraded":"2024-03-14T22:05:39.317932+0000","last_fullsized":"2024-03-14T22:05:39.317932+0000","mapping_epoch":18,"log_start":"23'1500","ondisk_log_start":"23'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11502","last_scrub_stamp":"2024-03-14T22:05:39.317814+0000","last_deep_scrub":"23'11502","last_deep_scrub_stamp":"2024-03-14T22:05:39.317814+0000","last_clean_scrub_stamp":"2024-03-14T22:05:39.317814+0000","objects_scrubbed":4,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:23.951100+0000","scrub_duration":0.049637907000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11487,"num_write_kb":85212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'9516","reported_seq":9569,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:52.659321+0000","last_change":"2024-03-14T22:05:52.659529+0000","last_active":"2024-03-14T22:05:52.659321+0000","last_peered":"2024-03-14T22:05:52.659321+0000","last_clean":"2024-03-14T22:05:52.659321+0000","last_became_active":"2024-03-14T21:14:03.033604+0000","last_became_peered":"2024-03-14T21:14:03.033604+0000","last_unstale":"2024-03-14T22:05:52.659321+0000","last_undegraded":"2024-03-14T22:05:52.659321+0000","last_fullsized":"2024-03-14T22:05:52.659321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9516","last_scrub_stamp":"2024-03-14T22:05:52.659298+0000","last_deep_scrub":"23'9516","last_deep_scrub_stamp":"2024-03-14T22:05:52.659298+0000","last_clean_scrub_stamp":"2024-03-14T22:05:52.659298+0000","objects_scrubbed":0,"log_size":9516,"log_dups_size":0,"ondisk_log_size":9516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:11.394549+0000","scrub_duration":0.012300805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":7940,"num_write_kb":115635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'8906","reported_seq":8954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.143252+0000","last_change":"2024-03-14T22:05:34.143252+0000","last_active":"2024-03-14T22:05:34.143252+0000","last_peered":"2024-03-14T22:05:34.143252+0000","last_clean":"2024-03-14T22:05:34.143252+0000","last_became_active":"2024-03-14T21:14:02.653629+0000","last_became_peered":"2024-03-14T21:14:02.653629+0000","last_unstale":"2024-03-14T22:05:34.143252+0000","last_undegraded":"2024-03-14T22:05:34.143252+0000","last_fullsized":"2024-03-14T22:05:34.143252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8906","last_scrub_stamp":"2024-03-14T22:05:34.143115+0000","last_deep_scrub":"23'8906","last_deep_scrub_stamp":"2024-03-14T22:05:34.143115+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.143115+0000","objects_scrubbed":2,"log_size":8906,"log_dups_size":0,"ondisk_log_size":8906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:05.686307+0000","scrub_duration":0.021707378999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8892,"num_write_kb":67382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1170,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'9427","reported_seq":9478,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:09.484008+0000","last_change":"2024-03-14T22:06:09.484137+0000","last_active":"2024-03-14T22:06:09.484008+0000","last_peered":"2024-03-14T22:06:09.484008+0000","last_clean":"2024-03-14T22:06:09.484008+0000","last_became_active":"2024-03-14T21:14:03.031141+0000","last_became_peered":"2024-03-14T21:14:03.031141+0000","last_unstale":"2024-03-14T22:06:09.484008+0000","last_undegraded":"2024-03-14T22:06:09.484008+0000","last_fullsized":"2024-03-14T22:06:09.484008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9427","last_scrub_stamp":"2024-03-14T22:06:09.483985+0000","last_deep_scrub":"23'9427","last_deep_scrub_stamp":"2024-03-14T22:06:09.483985+0000","last_clean_scrub_stamp":"2024-03-14T22:06:09.483985+0000","objects_scrubbed":0,"log_size":9427,"log_dups_size":0,"ondisk_log_size":9427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T09:19:27.695841+0000","scrub_duration":0.011868826000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1018,"num_write":7820,"num_write_kb":110543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.280716+0000","last_change":"2024-03-14T22:05:38.280716+0000","last_active":"2024-03-14T22:05:38.280716+0000","last_peered":"2024-03-14T22:05:38.280716+0000","last_clean":"2024-03-14T22:05:38.280716+0000","last_became_active":"2024-03-14T21:13:52.634785+0000","last_became_peered":"2024-03-14T21:13:52.634785+0000","last_unstale":"2024-03-14T22:05:38.280716+0000","last_undegraded":"2024-03-14T22:05:38.280716+0000","last_fullsized":"2024-03-14T22:05:38.280716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-03-14T22:05:38.280592+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-03-14T22:05:38.280592+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.280592+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:12.487548+0000","scrub_duration":0.0062042419999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'8579","reported_seq":8629,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:32.525126+0000","last_change":"2024-03-14T22:05:32.525126+0000","last_active":"2024-03-14T22:05:32.525126+0000","last_peered":"2024-03-14T22:05:32.525126+0000","last_clean":"2024-03-14T22:05:32.525126+0000","last_became_active":"2024-03-14T21:14:02.652605+0000","last_became_peered":"2024-03-14T21:14:02.652605+0000","last_unstale":"2024-03-14T22:05:32.525126+0000","last_undegraded":"2024-03-14T22:05:32.525126+0000","last_fullsized":"2024-03-14T22:05:32.525126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8579","last_scrub_stamp":"2024-03-14T22:05:32.525005+0000","last_deep_scrub":"23'8579","last_deep_scrub_stamp":"2024-03-14T22:05:32.525005+0000","last_clean_scrub_stamp":"2024-03-14T22:05:32.525005+0000","objects_scrubbed":2,"log_size":8579,"log_dups_size":0,"ondisk_log_size":8579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:06.731961+0000","scrub_duration":0.031238887999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":67724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'11133","reported_seq":11185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:04.961103+0000","last_change":"2024-03-14T22:06:04.961103+0000","last_active":"2024-03-14T22:06:04.961103+0000","last_peered":"2024-03-14T22:06:04.961103+0000","last_clean":"2024-03-14T22:06:04.961103+0000","last_became_active":"2024-03-14T21:14:03.038611+0000","last_became_peered":"2024-03-14T21:14:03.038611+0000","last_unstale":"2024-03-14T22:06:04.961103+0000","last_undegraded":"2024-03-14T22:06:04.961103+0000","last_fullsized":"2024-03-14T22:06:04.961103+0000","mapping_epoch":19,"log_start":"23'1100","ondisk_log_start":"23'1100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11133","last_scrub_stamp":"2024-03-14T22:06:04.960946+0000","last_deep_scrub":"23'11133","last_deep_scrub_stamp":"2024-03-14T22:06:04.960946+0000","last_clean_scrub_stamp":"2024-03-14T22:06:04.960946+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:23:24.994219+0000","scrub_duration":0.013835398,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":9507,"num_write_kb":117178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'8677","reported_seq":8728,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:52.523012+0000","last_change":"2024-03-14T22:05:52.523012+0000","last_active":"2024-03-14T22:05:52.523012+0000","last_peered":"2024-03-14T22:05:52.523012+0000","last_clean":"2024-03-14T22:05:52.523012+0000","last_became_active":"2024-03-14T21:14:02.660952+0000","last_became_peered":"2024-03-14T21:14:02.660952+0000","last_unstale":"2024-03-14T22:05:52.523012+0000","last_undegraded":"2024-03-14T22:05:52.523012+0000","last_fullsized":"2024-03-14T22:05:52.523012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8677","last_scrub_stamp":"2024-03-14T22:05:52.522865+0000","last_deep_scrub":"23'8677","last_deep_scrub_stamp":"2024-03-14T22:05:52.522865+0000","last_clean_scrub_stamp":"2024-03-14T22:05:52.522865+0000","objects_scrubbed":1,"log_size":8677,"log_dups_size":0,"ondisk_log_size":8677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:51.822151+0000","scrub_duration":0.0171749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8672,"num_write_kb":67194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'10246","reported_seq":10297,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.131437+0000","last_change":"2024-03-14T22:05:42.131437+0000","last_active":"2024-03-14T22:05:42.131437+0000","last_peered":"2024-03-14T22:05:42.131437+0000","last_clean":"2024-03-14T22:05:42.131437+0000","last_became_active":"2024-03-14T21:14:03.032895+0000","last_became_peered":"2024-03-14T21:14:03.032895+0000","last_unstale":"2024-03-14T22:05:42.131437+0000","last_undegraded":"2024-03-14T22:05:42.131437+0000","last_fullsized":"2024-03-14T22:05:42.131437+0000","mapping_epoch":19,"log_start":"23'200","ondisk_log_start":"23'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10246","last_scrub_stamp":"2024-03-14T22:05:42.131282+0000","last_deep_scrub":"23'10246","last_deep_scrub_stamp":"2024-03-14T22:05:42.131282+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.131282+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:13.674474+0000","scrub_duration":0.011884756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":8656,"num_write_kb":115905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11306","reported_seq":12023,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:35.173769+0000","last_change":"2024-03-14T22:05:35.173769+0000","last_active":"2024-03-14T22:05:35.173769+0000","last_peered":"2024-03-14T22:05:35.173769+0000","last_clean":"2024-03-14T22:05:35.173769+0000","last_became_active":"2024-03-14T21:14:02.658224+0000","last_became_peered":"2024-03-14T21:14:02.658224+0000","last_unstale":"2024-03-14T22:05:35.173769+0000","last_undegraded":"2024-03-14T22:05:35.173769+0000","last_fullsized":"2024-03-14T22:05:35.173769+0000","mapping_epoch":18,"log_start":"23'1300","ondisk_log_start":"23'1300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11306","last_scrub_stamp":"2024-03-14T22:05:35.173652+0000","last_deep_scrub":"23'11306","last_deep_scrub_stamp":"2024-03-14T22:05:35.173652+0000","last_clean_scrub_stamp":"2024-03-14T22:05:35.173652+0000","objects_scrubbed":3,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:26:19.806939+0000","scrub_duration":0.046066214000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":669,"num_read_kb":4516,"num_write":11296,"num_write_kb":89038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'9443","reported_seq":9496,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:02.462769+0000","last_change":"2024-03-14T22:06:02.462769+0000","last_active":"2024-03-14T22:06:02.462769+0000","last_peered":"2024-03-14T22:06:02.462769+0000","last_clean":"2024-03-14T22:06:02.462769+0000","last_became_active":"2024-03-14T21:14:03.034747+0000","last_became_peered":"2024-03-14T21:14:03.034747+0000","last_unstale":"2024-03-14T22:06:02.462769+0000","last_undegraded":"2024-03-14T22:06:02.462769+0000","last_fullsized":"2024-03-14T22:06:02.462769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9443","last_scrub_stamp":"2024-03-14T22:06:02.462621+0000","last_deep_scrub":"23'9443","last_deep_scrub_stamp":"2024-03-14T22:06:02.462621+0000","last_clean_scrub_stamp":"2024-03-14T22:06:02.462621+0000","objects_scrubbed":0,"log_size":9443,"log_dups_size":0,"ondisk_log_size":9443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:55.841618+0000","scrub_duration":0.011137756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":764,"num_write":7837,"num_write_kb":123513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'9296","reported_seq":9346,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.595654+0000","last_change":"2024-03-14T22:05:38.595654+0000","last_active":"2024-03-14T22:05:38.595654+0000","last_peered":"2024-03-14T22:05:38.595654+0000","last_clean":"2024-03-14T22:05:38.595654+0000","last_became_active":"2024-03-14T21:14:02.657130+0000","last_became_peered":"2024-03-14T21:14:02.657130+0000","last_unstale":"2024-03-14T22:05:38.595654+0000","last_undegraded":"2024-03-14T22:05:38.595654+0000","last_fullsized":"2024-03-14T22:05:38.595654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9296","last_scrub_stamp":"2024-03-14T22:05:38.595537+0000","last_deep_scrub":"23'9296","last_deep_scrub_stamp":"2024-03-14T22:05:38.595537+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.595537+0000","objects_scrubbed":0,"log_size":9296,"log_dups_size":0,"ondisk_log_size":9296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:20.624547+0000","scrub_duration":0.0060746669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9294,"num_write_kb":71830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'9546","reported_seq":9596,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.855707+0000","last_change":"2024-03-14T22:05:43.855707+0000","last_active":"2024-03-14T22:05:43.855707+0000","last_peered":"2024-03-14T22:05:43.855707+0000","last_clean":"2024-03-14T22:05:43.855707+0000","last_became_active":"2024-03-14T21:14:03.035748+0000","last_became_peered":"2024-03-14T21:14:03.035748+0000","last_unstale":"2024-03-14T22:05:43.855707+0000","last_undegraded":"2024-03-14T22:05:43.855707+0000","last_fullsized":"2024-03-14T22:05:43.855707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9546","last_scrub_stamp":"2024-03-14T22:05:43.855569+0000","last_deep_scrub":"23'9546","last_deep_scrub_stamp":"2024-03-14T22:05:43.855569+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.855569+0000","objects_scrubbed":0,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T03:35:00.547431+0000","scrub_duration":0.01173973,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":256,"num_write":7936,"num_write_kb":109238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'11136","reported_seq":11186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:50.431205+0000","last_change":"2024-03-14T22:05:50.431205+0000","last_active":"2024-03-14T22:05:50.431205+0000","last_peered":"2024-03-14T22:05:50.431205+0000","last_clean":"2024-03-14T22:05:50.431205+0000","last_became_active":"2024-03-14T21:14:02.652794+0000","last_became_peered":"2024-03-14T21:14:02.652794+0000","last_unstale":"2024-03-14T22:05:50.431205+0000","last_undegraded":"2024-03-14T22:05:50.431205+0000","last_fullsized":"2024-03-14T22:05:50.431205+0000","mapping_epoch":18,"log_start":"23'1100","ondisk_log_start":"23'1100","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11136","last_scrub_stamp":"2024-03-14T22:05:50.431052+0000","last_deep_scrub":"23'11136","last_deep_scrub_stamp":"2024-03-14T22:05:50.431052+0000","last_clean_scrub_stamp":"2024-03-14T22:05:50.431052+0000","objects_scrubbed":2,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:32.460062+0000","scrub_duration":0.031083458000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11130,"num_write_kb":85841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'12171","reported_seq":12220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:50.718554+0000","last_change":"2024-03-14T22:05:50.718554+0000","last_active":"2024-03-14T22:05:50.718554+0000","last_peered":"2024-03-14T22:05:50.718554+0000","last_clean":"2024-03-14T22:05:50.718554+0000","last_became_active":"2024-03-14T21:14:03.031154+0000","last_became_peered":"2024-03-14T21:14:03.031154+0000","last_unstale":"2024-03-14T22:05:50.718554+0000","last_undegraded":"2024-03-14T22:05:50.718554+0000","last_fullsized":"2024-03-14T22:05:50.718554+0000","mapping_epoch":19,"log_start":"23'2100","ondisk_log_start":"23'2100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12171","last_scrub_stamp":"2024-03-14T22:05:50.718411+0000","last_deep_scrub":"23'12171","last_deep_scrub_stamp":"2024-03-14T22:05:50.718411+0000","last_clean_scrub_stamp":"2024-03-14T22:05:50.718411+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:44.097408+0000","scrub_duration":0.014855348000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10560,"num_write_kb":124816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'9284","reported_seq":9334,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.431939+0000","last_change":"2024-03-14T22:05:47.431939+0000","last_active":"2024-03-14T22:05:47.431939+0000","last_peered":"2024-03-14T22:05:47.431939+0000","last_clean":"2024-03-14T22:05:47.431939+0000","last_became_active":"2024-03-14T21:14:02.659270+0000","last_became_peered":"2024-03-14T21:14:02.659270+0000","last_unstale":"2024-03-14T22:05:47.431939+0000","last_undegraded":"2024-03-14T22:05:47.431939+0000","last_fullsized":"2024-03-14T22:05:47.431939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9284","last_scrub_stamp":"2024-03-14T22:05:47.431780+0000","last_deep_scrub":"23'9284","last_deep_scrub_stamp":"2024-03-14T22:05:47.431780+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.431780+0000","objects_scrubbed":1,"log_size":9284,"log_dups_size":0,"ondisk_log_size":9284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:37.454481+0000","scrub_duration":0.019347982999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9294,"num_write_kb":71728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'9772","reported_seq":9826,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:57.488903+0000","last_change":"2024-03-14T22:05:57.488903+0000","last_active":"2024-03-14T22:05:57.488903+0000","last_peered":"2024-03-14T22:05:57.488903+0000","last_clean":"2024-03-14T22:05:57.488903+0000","last_became_active":"2024-03-14T21:14:03.022333+0000","last_became_peered":"2024-03-14T21:14:03.022333+0000","last_unstale":"2024-03-14T22:05:57.488903+0000","last_undegraded":"2024-03-14T22:05:57.488903+0000","last_fullsized":"2024-03-14T22:05:57.488903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9772","last_scrub_stamp":"2024-03-14T22:05:57.488748+0000","last_deep_scrub":"23'9772","last_deep_scrub_stamp":"2024-03-14T22:05:57.488748+0000","last_clean_scrub_stamp":"2024-03-14T22:05:57.488748+0000","objects_scrubbed":0,"log_size":9772,"log_dups_size":0,"ondisk_log_size":9772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:56.788033+0000","scrub_duration":0.013426983,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1272,"num_write":8091,"num_write_kb":107879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'8360","reported_seq":9002,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:38.381548+0000","last_change":"2024-03-14T22:05:38.381548+0000","last_active":"2024-03-14T22:05:38.381548+0000","last_peered":"2024-03-14T22:05:38.381548+0000","last_clean":"2024-03-14T22:05:38.381548+0000","last_became_active":"2024-03-14T21:14:02.661864+0000","last_became_peered":"2024-03-14T21:14:02.661864+0000","last_unstale":"2024-03-14T22:05:38.381548+0000","last_undegraded":"2024-03-14T22:05:38.381548+0000","last_fullsized":"2024-03-14T22:05:38.381548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8360","last_scrub_stamp":"2024-03-14T22:05:38.381407+0000","last_deep_scrub":"23'8360","last_deep_scrub_stamp":"2024-03-14T22:05:38.381407+0000","last_clean_scrub_stamp":"2024-03-14T22:05:38.381407+0000","objects_scrubbed":2,"log_size":8360,"log_dups_size":0,"ondisk_log_size":8360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:31:09.924599+0000","scrub_duration":0.064290047000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":593,"num_read_kb":4491,"num_write":9534,"num_write_kb":73072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'10081","reported_seq":10138,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:42.020373+0000","last_change":"2024-03-14T22:05:42.020373+0000","last_active":"2024-03-14T22:05:42.020373+0000","last_peered":"2024-03-14T22:05:42.020373+0000","last_clean":"2024-03-14T22:05:42.020373+0000","last_became_active":"2024-03-14T21:14:03.037684+0000","last_became_peered":"2024-03-14T21:14:03.037684+0000","last_unstale":"2024-03-14T22:05:42.020373+0000","last_undegraded":"2024-03-14T22:05:42.020373+0000","last_fullsized":"2024-03-14T22:05:42.020373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10081","last_scrub_stamp":"2024-03-14T22:05:42.020207+0000","last_deep_scrub":"23'10081","last_deep_scrub_stamp":"2024-03-14T22:05:42.020207+0000","last_clean_scrub_stamp":"2024-03-14T22:05:42.020207+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:16.227164+0000","scrub_duration":0.010075914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2288,"num_write":8532,"num_write_kb":114847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5882","reported_seq":5931,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.478496+0000","last_change":"2024-03-14T22:05:47.478496+0000","last_active":"2024-03-14T22:05:47.478496+0000","last_peered":"2024-03-14T22:05:47.478496+0000","last_clean":"2024-03-14T22:05:47.478496+0000","last_became_active":"2024-03-14T21:14:02.665495+0000","last_became_peered":"2024-03-14T21:14:02.665495+0000","last_unstale":"2024-03-14T22:05:47.478496+0000","last_undegraded":"2024-03-14T22:05:47.478496+0000","last_fullsized":"2024-03-14T22:05:47.478496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5882","last_scrub_stamp":"2024-03-14T22:05:47.478380+0000","last_deep_scrub":"23'5882","last_deep_scrub_stamp":"2024-03-14T22:05:47.478380+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.478380+0000","objects_scrubbed":2,"log_size":5882,"log_dups_size":0,"ondisk_log_size":5882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:21.685337+0000","scrub_duration":0.023401211000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5885,"num_write_kb":44672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17052,"num_omap_keys":21,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'10915","reported_seq":10970,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:47.120809+0000","last_change":"2024-03-14T22:05:47.120809+0000","last_active":"2024-03-14T22:05:47.120809+0000","last_peered":"2024-03-14T22:05:47.120809+0000","last_clean":"2024-03-14T22:05:47.120809+0000","last_became_active":"2024-03-14T21:14:03.028991+0000","last_became_peered":"2024-03-14T21:14:03.028991+0000","last_unstale":"2024-03-14T22:05:47.120809+0000","last_undegraded":"2024-03-14T22:05:47.120809+0000","last_fullsized":"2024-03-14T22:05:47.120809+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10915","last_scrub_stamp":"2024-03-14T22:05:47.120658+0000","last_deep_scrub":"23'10915","last_deep_scrub_stamp":"2024-03-14T22:05:47.120658+0000","last_clean_scrub_stamp":"2024-03-14T22:05:47.120658+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:21.327615+0000","scrub_duration":0.011394476000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1786,"num_write":9274,"num_write_kb":112928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'4363","reported_seq":4412,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:56.577952+0000","last_change":"2024-03-14T22:05:56.577952+0000","last_active":"2024-03-14T22:05:56.577952+0000","last_peered":"2024-03-14T22:05:56.577952+0000","last_clean":"2024-03-14T22:05:56.577952+0000","last_became_active":"2024-03-14T21:14:02.661400+0000","last_became_peered":"2024-03-14T21:14:02.661400+0000","last_unstale":"2024-03-14T22:05:56.577952+0000","last_undegraded":"2024-03-14T22:05:56.577952+0000","last_fullsized":"2024-03-14T22:05:56.577952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4363","last_scrub_stamp":"2024-03-14T22:05:56.577822+0000","last_deep_scrub":"23'4363","last_deep_scrub_stamp":"2024-03-14T22:05:56.577822+0000","last_clean_scrub_stamp":"2024-03-14T22:05:56.577822+0000","objects_scrubbed":3,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:43:07.362865+0000","scrub_duration":0.035695787999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4362,"num_write_kb":43693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'9887","reported_seq":9948,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:40.803829+0000","last_change":"2024-03-14T22:05:40.803829+0000","last_active":"2024-03-14T22:05:40.803829+0000","last_peered":"2024-03-14T22:05:40.803829+0000","last_clean":"2024-03-14T22:05:40.803829+0000","last_became_active":"2024-03-14T21:14:03.039299+0000","last_became_peered":"2024-03-14T21:14:03.039299+0000","last_unstale":"2024-03-14T22:05:40.803829+0000","last_undegraded":"2024-03-14T22:05:40.803829+0000","last_fullsized":"2024-03-14T22:05:40.803829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9887","last_scrub_stamp":"2024-03-14T22:05:40.803679+0000","last_deep_scrub":"23'9887","last_deep_scrub_stamp":"2024-03-14T22:05:40.803679+0000","last_clean_scrub_stamp":"2024-03-14T22:05:40.803679+0000","objects_scrubbed":0,"log_size":9887,"log_dups_size":0,"ondisk_log_size":9887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:46:25.442439+0000","scrub_duration":0.011556728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3306,"num_write":8266,"num_write_kb":116331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'6408","reported_seq":6457,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:43.393658+0000","last_change":"2024-03-14T22:05:43.393658+0000","last_active":"2024-03-14T22:05:43.393658+0000","last_peered":"2024-03-14T22:05:43.393658+0000","last_clean":"2024-03-14T22:05:43.393658+0000","last_became_active":"2024-03-14T21:14:02.660050+0000","last_became_peered":"2024-03-14T21:14:02.660050+0000","last_unstale":"2024-03-14T22:05:43.393658+0000","last_undegraded":"2024-03-14T22:05:43.393658+0000","last_fullsized":"2024-03-14T22:05:43.393658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6408","last_scrub_stamp":"2024-03-14T22:05:43.393505+0000","last_deep_scrub":"23'6408","last_deep_scrub_stamp":"2024-03-14T22:05:43.393505+0000","last_clean_scrub_stamp":"2024-03-14T22:05:43.393505+0000","objects_scrubbed":2,"log_size":6408,"log_dups_size":0,"ondisk_log_size":6408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:37:17.600461+0000","scrub_duration":0.029677097,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":6417,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'10927","reported_seq":10975,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:48.972164+0000","last_change":"2024-03-14T22:05:48.972164+0000","last_active":"2024-03-14T22:05:48.972164+0000","last_peered":"2024-03-14T22:05:48.972164+0000","last_clean":"2024-03-14T22:05:48.972164+0000","last_became_active":"2024-03-14T21:14:03.040583+0000","last_became_peered":"2024-03-14T21:14:03.040583+0000","last_unstale":"2024-03-14T22:05:48.972164+0000","last_undegraded":"2024-03-14T22:05:48.972164+0000","last_fullsized":"2024-03-14T22:05:48.972164+0000","mapping_epoch":19,"log_start":"23'900","ondisk_log_start":"23'900","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10927","last_scrub_stamp":"2024-03-14T22:05:48.972019+0000","last_deep_scrub":"23'10927","last_deep_scrub_stamp":"2024-03-14T22:05:48.972019+0000","last_clean_scrub_stamp":"2024-03-14T22:05:48.972019+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:31.001029+0000","scrub_duration":0.013693627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":120976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'9353","reported_seq":9402,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:34.295195+0000","last_change":"2024-03-14T22:05:34.295195+0000","last_active":"2024-03-14T22:05:34.295195+0000","last_peered":"2024-03-14T22:05:34.295195+0000","last_clean":"2024-03-14T22:05:34.295195+0000","last_became_active":"2024-03-14T21:14:02.660789+0000","last_became_peered":"2024-03-14T21:14:02.660789+0000","last_unstale":"2024-03-14T22:05:34.295195+0000","last_undegraded":"2024-03-14T22:05:34.295195+0000","last_fullsized":"2024-03-14T22:05:34.295195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9353","last_scrub_stamp":"2024-03-14T22:05:34.295063+0000","last_deep_scrub":"23'9353","last_deep_scrub_stamp":"2024-03-14T22:05:34.295063+0000","last_clean_scrub_stamp":"2024-03-14T22:05:34.295063+0000","objects_scrubbed":1,"log_size":9353,"log_dups_size":0,"ondisk_log_size":9353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:46.557728+0000","scrub_duration":0.019716635999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9374,"num_write_kb":71713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'10198","reported_seq":10249,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:55.934953+0000","last_change":"2024-03-14T22:05:55.935083+0000","last_active":"2024-03-14T22:05:55.934953+0000","last_peered":"2024-03-14T22:05:55.934953+0000","last_clean":"2024-03-14T22:05:55.934953+0000","last_became_active":"2024-03-14T21:14:03.036302+0000","last_became_peered":"2024-03-14T21:14:03.036302+0000","last_unstale":"2024-03-14T22:05:55.934953+0000","last_undegraded":"2024-03-14T22:05:55.934953+0000","last_fullsized":"2024-03-14T22:05:55.934953+0000","mapping_epoch":19,"log_start":"23'100","ondisk_log_start":"23'100","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10198","last_scrub_stamp":"2024-03-14T22:05:55.934931+0000","last_deep_scrub":"23'10198","last_deep_scrub_stamp":"2024-03-14T22:05:55.934931+0000","last_clean_scrub_stamp":"2024-03-14T22:05:55.934931+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:27:55.234216+0000","scrub_duration":0.010658134999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":8519,"num_write_kb":106953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'11405","reported_seq":11854,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:46.439556+0000","last_change":"2024-03-14T22:05:46.439556+0000","last_active":"2024-03-14T22:05:46.439556+0000","last_peered":"2024-03-14T22:05:46.439556+0000","last_clean":"2024-03-14T22:05:46.439556+0000","last_became_active":"2024-03-14T21:14:02.662826+0000","last_became_peered":"2024-03-14T21:14:02.662826+0000","last_unstale":"2024-03-14T22:05:46.439556+0000","last_undegraded":"2024-03-14T22:05:46.439556+0000","last_fullsized":"2024-03-14T22:05:46.439556+0000","mapping_epoch":18,"log_start":"23'1400","ondisk_log_start":"23'1400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11405","last_scrub_stamp":"2024-03-14T22:05:46.439398+0000","last_deep_scrub":"23'11405","last_deep_scrub_stamp":"2024-03-14T22:05:46.439398+0000","last_clean_scrub_stamp":"2024-03-14T22:05:46.439398+0000","objects_scrubbed":5,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T08:58:17.165906+0000","scrub_duration":0.054388652000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13689588,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":402,"num_read_kb":1363,"num_write":11407,"num_write_kb":86544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":368,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'13463","reported_seq":13517,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:10.754779+0000","last_change":"2024-03-14T22:06:10.754779+0000","last_active":"2024-03-14T22:06:10.754779+0000","last_peered":"2024-03-14T22:06:10.754779+0000","last_clean":"2024-03-14T22:06:10.754779+0000","last_became_active":"2024-03-14T21:14:03.039614+0000","last_became_peered":"2024-03-14T21:14:03.039614+0000","last_unstale":"2024-03-14T22:06:10.754779+0000","last_undegraded":"2024-03-14T22:06:10.754779+0000","last_fullsized":"2024-03-14T22:06:10.754779+0000","mapping_epoch":19,"log_start":"23'3400","ondisk_log_start":"23'3400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13463","last_scrub_stamp":"2024-03-14T22:06:10.754640+0000","last_deep_scrub":"23'13463","last_deep_scrub_stamp":"2024-03-14T22:06:10.754640+0000","last_clean_scrub_stamp":"2024-03-14T22:06:10.754640+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T00:43:33.802681+0000","scrub_duration":0.016388162000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1526,"num_write":11847,"num_write_kb":149794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'10677","reported_seq":10722,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:54.646809+0000","last_change":"2024-03-14T22:05:54.646938+0000","last_active":"2024-03-14T22:05:54.646809+0000","last_peered":"2024-03-14T22:05:54.646809+0000","last_clean":"2024-03-14T22:05:54.646809+0000","last_became_active":"2024-03-14T21:14:03.034616+0000","last_became_peered":"2024-03-14T21:14:03.034616+0000","last_unstale":"2024-03-14T22:05:54.646809+0000","last_undegraded":"2024-03-14T22:05:54.646809+0000","last_fullsized":"2024-03-14T22:05:54.646809+0000","mapping_epoch":19,"log_start":"23'600","ondisk_log_start":"23'600","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10677","last_scrub_stamp":"2024-03-14T22:05:54.646787+0000","last_deep_scrub":"23'10677","last_deep_scrub_stamp":"2024-03-14T22:05:54.646787+0000","last_clean_scrub_stamp":"2024-03-14T22:05:54.646787+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T23:23:14.680060+0000","scrub_duration":0.011557494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":8986,"num_write_kb":111494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6869","reported_seq":6914,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:45.414748+0000","last_change":"2024-03-14T22:05:45.414869+0000","last_active":"2024-03-14T22:05:45.414748+0000","last_peered":"2024-03-14T22:05:45.414748+0000","last_clean":"2024-03-14T22:05:45.414748+0000","last_became_active":"2024-03-14T21:14:02.660077+0000","last_became_peered":"2024-03-14T21:14:02.660077+0000","last_unstale":"2024-03-14T22:05:45.414748+0000","last_undegraded":"2024-03-14T22:05:45.414748+0000","last_fullsized":"2024-03-14T22:05:45.414748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6869","last_scrub_stamp":"2024-03-14T22:05:45.414726+0000","last_deep_scrub":"23'6869","last_deep_scrub_stamp":"2024-03-14T22:05:45.414726+0000","last_clean_scrub_stamp":"2024-03-14T22:05:45.414726+0000","objects_scrubbed":5,"log_size":6869,"log_dups_size":0,"ondisk_log_size":6869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:05:27.443736+0000","scrub_duration":0.057950375999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6885,"num_write_kb":66562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'11249","reported_seq":11298,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:48.149731+0000","last_change":"2024-03-14T22:05:48.149866+0000","last_active":"2024-03-14T22:05:48.149731+0000","last_peered":"2024-03-14T22:05:48.149731+0000","last_clean":"2024-03-14T22:05:48.149731+0000","last_became_active":"2024-03-14T21:14:44.859128+0000","last_became_peered":"2024-03-14T21:14:44.859128+0000","last_unstale":"2024-03-14T22:05:48.149731+0000","last_undegraded":"2024-03-14T22:05:48.149731+0000","last_fullsized":"2024-03-14T22:05:48.149731+0000","mapping_epoch":22,"log_start":"23'1200","ondisk_log_start":"23'1200","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11249","last_scrub_stamp":"2024-03-14T22:05:48.149707+0000","last_deep_scrub":"23'11249","last_deep_scrub_stamp":"2024-03-14T22:05:48.149707+0000","last_clean_scrub_stamp":"2024-03-14T22:05:48.149707+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:34:38.172409+0000","scrub_duration":0.015019927000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1780,"num_write":9572,"num_write_kb":116659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"23'15428","reported_seq":16542,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:57.600392+0000","last_change":"2024-03-14T22:05:57.600521+0000","last_active":"2024-03-14T22:05:57.600392+0000","last_peered":"2024-03-14T22:05:57.600392+0000","last_clean":"2024-03-14T22:05:57.600392+0000","last_became_active":"2024-03-14T21:14:02.662456+0000","last_became_peered":"2024-03-14T21:14:02.662456+0000","last_unstale":"2024-03-14T22:05:57.600392+0000","last_undegraded":"2024-03-14T22:05:57.600392+0000","last_fullsized":"2024-03-14T22:05:57.600392+0000","mapping_epoch":18,"log_start":"23'5400","ondisk_log_start":"23'5400","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'15428","last_scrub_stamp":"2024-03-14T22:05:57.600372+0000","last_deep_scrub":"23'15428","last_deep_scrub_stamp":"2024-03-14T22:05:57.600372+0000","last_clean_scrub_stamp":"2024-03-14T22:05:57.600372+0000","objects_scrubbed":6,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:50.979369+0000","scrub_duration":0.079319829999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1069,"num_read_kb":4840,"num_write":15404,"num_write_kb":112501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'10432","reported_seq":10492,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:06.161157+0000","last_change":"2024-03-14T22:06:06.161157+0000","last_active":"2024-03-14T22:06:06.161157+0000","last_peered":"2024-03-14T22:06:06.161157+0000","last_clean":"2024-03-14T22:06:06.161157+0000","last_became_active":"2024-03-14T21:14:03.036537+0000","last_became_peered":"2024-03-14T21:14:03.036537+0000","last_unstale":"2024-03-14T22:06:06.161157+0000","last_undegraded":"2024-03-14T22:06:06.161157+0000","last_fullsized":"2024-03-14T22:06:06.161157+0000","mapping_epoch":19,"log_start":"23'400","ondisk_log_start":"23'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10432","last_scrub_stamp":"2024-03-14T22:06:06.160892+0000","last_deep_scrub":"23'10432","last_deep_scrub_stamp":"2024-03-14T22:06:06.160892+0000","last_clean_scrub_stamp":"2024-03-14T22:06:06.160892+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T22:55:59.539890+0000","scrub_duration":0.005531083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2796,"num_write":8857,"num_write_kb":109660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'8707","reported_seq":8758,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:33.083418+0000","last_change":"2024-03-14T22:05:33.083418+0000","last_active":"2024-03-14T22:05:33.083418+0000","last_peered":"2024-03-14T22:05:33.083418+0000","last_clean":"2024-03-14T22:05:33.083418+0000","last_became_active":"2024-03-14T21:14:02.654680+0000","last_became_peered":"2024-03-14T21:14:02.654680+0000","last_unstale":"2024-03-14T22:05:33.083418+0000","last_undegraded":"2024-03-14T22:05:33.083418+0000","last_fullsized":"2024-03-14T22:05:33.083418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8707","last_scrub_stamp":"2024-03-14T22:05:33.083300+0000","last_deep_scrub":"23'8707","last_deep_scrub_stamp":"2024-03-14T22:05:33.083300+0000","last_clean_scrub_stamp":"2024-03-14T22:05:33.083300+0000","objects_scrubbed":1,"log_size":8707,"log_dups_size":0,"ondisk_log_size":8707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T01:44:45.345964+0000","scrub_duration":0.015359457999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8698,"num_write_kb":67327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'11837","reported_seq":11888,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:06:02.952055+0000","last_change":"2024-03-14T22:06:02.952055+0000","last_active":"2024-03-14T22:06:02.952055+0000","last_peered":"2024-03-14T22:06:02.952055+0000","last_clean":"2024-03-14T22:06:02.952055+0000","last_became_active":"2024-03-14T21:14:03.032195+0000","last_became_peered":"2024-03-14T21:14:03.032195+0000","last_unstale":"2024-03-14T22:06:02.952055+0000","last_undegraded":"2024-03-14T22:06:02.952055+0000","last_fullsized":"2024-03-14T22:06:02.952055+0000","mapping_epoch":19,"log_start":"23'1800","ondisk_log_start":"23'1800","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11837","last_scrub_stamp":"2024-03-14T22:06:02.951924+0000","last_deep_scrub":"23'11837","last_deep_scrub_stamp":"2024-03-14T22:06:02.951924+0000","last_clean_scrub_stamp":"2024-03-14T22:06:02.951924+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T04:18:21.687175+0000","scrub_duration":0.01245275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":254,"num_write":10179,"num_write_kb":107836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'6051","reported_seq":6101,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-14T22:05:31.157143+0000","last_change":"2024-03-14T22:05:31.157143+0000","last_active":"2024-03-14T22:05:31.157143+0000","last_peered":"2024-03-14T22:05:31.157143+0000","last_clean":"2024-03-14T22:05:31.157143+0000","last_became_active":"2024-03-14T21:14:02.655643+0000","last_became_peered":"2024-03-14T21:14:02.655643+0000","last_unstale":"2024-03-14T22:05:31.157143+0000","last_undegraded":"2024-03-14T22:05:31.157143+0000","last_fullsized":"2024-03-14T22:05:31.157143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6051","last_scrub_stamp":"2024-03-14T22:05:31.157033+0000","last_deep_scrub":"23'6051","last_deep_scrub_stamp":"2024-03-14T22:05:31.157033+0000","last_clean_scrub_stamp":"2024-03-14T22:05:31.157033+0000","objects_scrubbed":5,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T05:11:59.042596+0000","scrub_duration":0.052529798000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6048,"num_write_kb":48854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":345,"num_read_kb":87814,"num_write":552757,"num_write_kb":7123402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":624278,"ondisk_log_size":624278,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":256907613,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":3941,"num_read_kb":24159,"num_write":290960,"num_write_kb":2282127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23590,"num_omap_keys":34,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":513884160,"data_stored":513815226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271687,"ondisk_log_size":271687,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575484,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252664,"kb_used_data":112496,"kb_used_omap":1,"kb_used_meta":1140158,"kb_avail":93119176,"statfs":{"total":96636764160,"available":95354036224,"internally_reserved":0,"allocated":115195904,"data_stored":115031593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1167522059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44600000000000001,"15min":0.48899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.19500000000000001},"max":{"1min":0.879,"5min":2.782,"15min":4.7999999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.63100000000000001,"15min":0.53200000000000003},"min":{"1min":0.29199999999999998,"5min":0.25,"15min":0.20999999999999999},"max":{"1min":4.8860000000000001,"5min":4.8860000000000001,"15min":4.8860000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.51300000000000001,"15min":0.505},"min":{"1min":0.248,"5min":0.248,"15min":0.23400000000000001},"max":{"1min":4.8410000000000002,"5min":4.8410000000000002,"15min":4.8410000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.28599999999999998,"15min":0.26400000000000001},"max":{"1min":2.2240000000000002,"5min":6.1440000000000001,"15min":6.1440000000000001},"last":0.622}]},{"osd":2,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52500000000000002,"15min":0.53200000000000003},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.25900000000000001},"max":{"1min":0.80300000000000005,"5min":3.8210000000000002,"15min":4.8310000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.63800000000000001,"15min":0.55300000000000005},"min":{"1min":0.36199999999999999,"5min":0.309,"15min":0.25900000000000001},"max":{"1min":4.1669999999999998,"5min":6.5780000000000003,"15min":6.5780000000000003},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.2519999999999998,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":3.165,"5min":3.165,"15min":4.851},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.254,"5min":0.23400000000000001,"15min":0.22500000000000001},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":5.0590000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.41399999999999998,"15min":0.44900000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.23300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":5.2679999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.878,"5min":1.3089999999999999,"15min":3.5760000000000001},"last":0.499},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.51800000000000002,"15min":0.54500000000000004},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":6.4790000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.27800000000000002},"max":{"1min":1.153,"5min":1.2529999999999999,"15min":4.157},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.39500000000000002,"5min":0.307,"15min":0.29299999999999998},"max":{"1min":2.403,"5min":2.403,"15min":5.0880000000000001},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101820,"kb_used_data":69876,"kb_used_omap":1,"kb_used_meta":1031934,"kb_avail":93270020,"statfs":{"total":96636764160,"available":95508500480,"internally_reserved":0,"allocated":71553024,"data_stored":71391599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1056700683},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.49199999999999999,"15min":0.58099999999999996},"min":{"1min":0.35599999999999998,"5min":0.222,"15min":0.188},"max":{"1min":0.77700000000000002,"5min":2.899,"15min":17.885999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.53300000000000003},"min":{"1min":0.435,"5min":0.25700000000000001,"15min":0.23000000000000001},"max":{"1min":0.745,"5min":6.1449999999999996,"15min":6.1449999999999996},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.503,"15min":0.52700000000000002},"min":{"1min":0.45500000000000002,"5min":0.26800000000000002,"15min":0.216},"max":{"1min":0.73099999999999998,"5min":3.3330000000000002,"15min":7.226},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.50900000000000001,"15min":0.51800000000000002},"min":{"1min":0.41599999999999998,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.81100000000000005,"5min":1.224,"15min":7.1699999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.48999999999999999,"5min":0.26600000000000001,"15min":0.25900000000000001},"max":{"1min":0.76400000000000001,"5min":6.1210000000000004,"15min":6.6189999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.42099999999999999,"5min":0.30599999999999999,"15min":0.26800000000000002},"max":{"1min":0.75,"5min":3.282,"15min":7.2489999999999997},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54400000000000004,"15min":0.56899999999999995},"min":{"1min":0.55500000000000005,"5min":0.28100000000000003,"15min":0.26200000000000001},"max":{"1min":0.76000000000000001,"5min":1.2470000000000001,"15min":7.1470000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.60099999999999998},"min":{"1min":0.52900000000000003,"5min":0.32700000000000001,"15min":0.29899999999999999},"max":{"1min":0.79900000000000004,"5min":6.1660000000000004,"15min":6.1660000000000004},"last":0.629}]},{"osd":4,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.47499999999999998,"15min":0.439},"min":{"1min":0.373,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":0.81399999999999995,"5min":2.468,"15min":7.3330000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51700000000000002,"15min":0.46400000000000002},"min":{"1min":0.40000000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.79200000000000004,"5min":6.0229999999999997,"15min":6.0229999999999997},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55100000000000005,"15min":0.497},"min":{"1min":0.42499999999999999,"5min":0.25600000000000001,"15min":0.254},"max":{"1min":0.77900000000000003,"5min":8.234,"15min":8.234},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.44,"5min":0.31,"15min":0.26900000000000002},"max":{"1min":0.85599999999999998,"5min":1.3340000000000001,"15min":7.5380000000000003},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.50800000000000001},"min":{"1min":0.46800000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82999999999999996,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.443},{"interface":"front","average":{"1min":0.628,"5min":0.54800000000000004,"15min":0.52500000000000002},"min":{"1min":0.51300000000000001,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.80800000000000005,"5min":3.3109999999999999,"15min":7.2039999999999997},"last":0.56899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575483,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1151736,"kb_used_data":69872,"kb_used_omap":1,"kb_used_meta":1081854,"kb_avail":93220104,"statfs":{"total":96636764160,"available":95457386496,"internally_reserved":0,"allocated":71548928,"data_stored":71391017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1107818763},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.54300000000000004},"min":{"1min":0.26200000000000001,"5min":0.223,"15min":0.19400000000000001},"max":{"1min":0.66100000000000003,"5min":3.2149999999999999,"15min":7.0990000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.497,"5min":0.53000000000000003,"15min":0.57299999999999995},"min":{"1min":0.26700000000000002,"5min":0.247,"15min":0.21099999999999999},"max":{"1min":0.73799999999999999,"5min":5.7359999999999998,"15min":12.281000000000001},"last":0.66800000000000004}]},{"osd":1,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.68799999999999994,"15min":0.57599999999999996},"min":{"1min":0.245,"5min":0.245,"15min":0.20599999999999999},"max":{"1min":0.63100000000000001,"5min":14.952,"15min":14.952},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.52200000000000002},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.247},"max":{"1min":0.745,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.75800000000000001,"15min":0.61599999999999999},"min":{"1min":0.29099999999999998,"5min":0.26600000000000001,"15min":0.251},"max":{"1min":0.70699999999999996,"5min":18.623000000000001,"15min":18.623000000000001},"last":0.625},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.96099999999999997,"15min":0.67300000000000004},"min":{"1min":0.32200000000000001,"5min":0.31,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":30.004999999999999,"15min":30.004999999999999},"last":0.68100000000000005}]},{"osd":3,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28000000000000003},"max":{"1min":0.69599999999999995,"5min":3.6600000000000001,"15min":9.2309999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.50600000000000001,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":0.82699999999999996,"15min":7.3780000000000001},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.59899999999999998,"15min":0.50700000000000001},"min":{"1min":0.23599999999999999,"5min":0.22700000000000001,"15min":0.20999999999999999},"max":{"1min":0.51900000000000002,"5min":12.414,"15min":12.414},"last":0.433},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.50800000000000001,"15min":0.54500000000000004},"min":{"1min":0.34100000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.66900000000000004,"5min":3.1970000000000001,"15min":8.3339999999999996},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45100000000000001,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.245,"15min":0.22},"max":{"1min":0.64000000000000001,"5min":1.905,"15min":7.4000000000000004},"last":0.497},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.621,"15min":0.55400000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.24299999999999999},"max":{"1min":0.63200000000000001,"5min":10.278,"15min":10.278},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Mar 14 22:05:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.27200000000000002,"15min":0.251},"max":{"1min":0.68500000000000005,"5min":3.262,"15min":7.1619999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54700000000000004,"15min":0.56000000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28399999999999997},"max":{"1min":0.72699999999999998,"5min":3.2909999999999999,"15min":7.3310000000000004},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":965948,"kb_used_data":42292,"kb_used_omap":1,"kb_used_meta":923646,"kb_avail":93405892,"statfs":{"total":96636764160,"available":95647633408,"internally_reserved":0,"allocated":43307008,"data_stored":43137601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":945813771},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.502,"15min":0.56200000000000006},"min":{"1min":0.41099999999999998,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":1.1319999999999999,"5min":2.23,"15min":15.013},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.20499999999999999},"max":{"1min":1.3120000000000001,"5min":2.1139999999999999,"15min":5.6879999999999997},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.254},"max":{"1min":1.3500000000000001,"5min":1.806,"15min":4.6479999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.48799999999999999,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.251,"15min":0.23200000000000001},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":4.0469999999999997},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.50700000000000001},"min":{"1min":0.29099999999999998,"5min":0.247,"15min":0.247},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":5.5540000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34399999999999997,"15min":0.29599999999999999},"max":{"1min":1.379,"5min":6.7329999999999997,"15min":6.7329999999999997},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.36599999999999999,"5min":0.30299999999999999,"15min":0.26700000000000002},"max":{"1min":2.8879999999999999,"5min":2.8879999999999999,"15min":9.9800000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.38,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.899},"last":0.745}]},{"osd":5,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.441,"15min":0.41899999999999998},"min":{"1min":0.25,"5min":0.219,"15min":0.219},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.0470000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.496,"15min":0.46700000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.22700000000000001},"max":{"1min":5.8860000000000001,"5min":5.8860000000000001,"15min":5.8860000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.245},"max":{"1min":2.21,"5min":2.21,"15min":3.8650000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.495,"15min":0.46899999999999997},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.24299999999999999},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.501,"15min":0.505},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23400000000000001},"max":{"1min":3.0190000000000001,"5min":3.0190000000000001,"15min":8.7639999999999993},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52700000000000002,"15min":0.502},"min":{"1min":0.34699999999999998,"5min":0.311,"15min":0.28399999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":3.7850000000000001},"last":0.77500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575483,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1135420,"kb_used_data":57588,"kb_used_omap":1,"kb_used_meta":1077822,"kb_avail":93236420,"statfs":{"total":96636764160,"available":95474094080,"internally_reserved":0,"allocated":58970112,"data_stored":58808687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":1103689995},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.17000000000000001},"max":{"1min":8.3350000000000009,"5min":8.3350000000000009,"15min":14.695},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.41699999999999998,"15min":0.52100000000000002},"min":{"1min":0.23599999999999999,"5min":0.19900000000000001,"15min":0.186},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":8.9649999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.41299999999999998,"15min":0.46000000000000002},"min":{"1min":0.22,"5min":0.215,"15min":0.19500000000000001},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":5.5289999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.55400000000000005,"15min":0.59199999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.20100000000000001},"max":{"1min":1.508,"5min":6.1740000000000004,"15min":14.898},"last":0.63}]},{"osd":2,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46300000000000002,"15min":0.56799999999999995},"min":{"1min":0.26300000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61599999999999999,"5min":3.7869999999999999,"15min":9.5150000000000006},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.28899999999999998,"5min":0.25800000000000001,"15min":0.23100000000000001},"max":{"1min":2.2090000000000001,"5min":2.9430000000000001,"15min":5.2859999999999996},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.74399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":0.84299999999999997,"5min":1.5229999999999999,"15min":14.071999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.60699999999999998,"15min":0.69899999999999995},"min":{"1min":0.40000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":14.109},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64400000000000002,"15min":0.68100000000000005},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.314},"max":{"1min":3.0209999999999999,"5min":5.1609999999999996,"15min":11.443},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.2589999999999999,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.377,"15min":0.27800000000000002},"max":{"1min":8.3659999999999997,"5min":8.3659999999999997,"15min":14.550000000000001},"last":0.755}]},{"osd":6,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65600000000000003,"15min":0.751},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.29299999999999998},"max":{"1min":2.1949999999999998,"5min":4.2539999999999996,"15min":9.7200000000000006},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.65700000000000003,"15min":0.70699999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35599999999999998},"max":{"1min":2.6360000000000001,"5min":2.8170000000000002,"15min":15.339},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Mar 14 22:05:33 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":8.2799999999999994,"5min":8.2799999999999994,"15min":14.726000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64200000000000002,"15min":0.76100000000000001},"min":{"1min":0.41399999999999998,"5min":0.379,"15min":0.33700000000000002},"max":{"1min":1.1180000000000001,"5min":2.9489999999999998,"15min":9.5079999999999991},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575484,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649796,"kb_used_data":45244,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":93722044,"statfs":{"total":96636764160,"available":95971373056,"internally_reserved":0,"allocated":46329856,"data_stored":46172026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":619051275},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.58199999999999996,"15min":0.497},"min":{"1min":0.217,"5min":0.185,"15min":0.182},"max":{"1min":6.3840000000000003,"5min":7.008,"15min":10.869},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.56799999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":3.8170000000000002,"5min":7.7699999999999996,"15min":12.317},"last":0.495}]},{"osd":1,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.59699999999999998,"15min":0.49099999999999999},"min":{"1min":0.25800000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":6.4249999999999998,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.20100000000000001},"max":{"1min":3.8769999999999998,"5min":7.8090000000000002,"15min":9.5429999999999993},"last":0.441}]},{"osd":3,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.63100000000000001},"min":{"1min":0.25600000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":2.5419999999999998,"5min":13.130000000000001,"15min":27.355},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.214,"5min":0.64700000000000002,"15min":0.62},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27500000000000002},"max":{"1min":7.5910000000000002,"5min":7.5910000000000002,"15min":12.451000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":0.90900000000000003,"15min":0.77200000000000002},"min":{"1min":0.46400000000000002,"5min":0.32100000000000001,"15min":0.28899999999999998},"max":{"1min":7.6239999999999997,"5min":8.0440000000000005,"15min":12.478999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.3289999999999997,"5min":7.9409999999999998,"15min":12.391999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.90200000000000002,"15min":0.76700000000000002},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27400000000000002},"max":{"1min":7.4089999999999998,"5min":13.318,"15min":20.84},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.5549999999999999,"5min":0.79300000000000004,"15min":0.73699999999999999},"min":{"1min":0.40799999999999997,"5min":0.35799999999999998,"15min":0.32000000000000001},"max":{"1min":12.319000000000001,"5min":12.319000000000001,"15min":12.319000000000001},"last":0.76800000000000002}]},{"osd":6,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.65400000000000003,"15min":0.66100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34300000000000003},"max":{"1min":2.6429999999999998,"5min":5.0209999999999999,"15min":6.2560000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.05,"5min":0.84899999999999998,"15min":0.68700000000000006},"min":{"1min":0.40100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.4420000000000002,"5min":13.177,"15min":13.177},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.74299999999999999,"15min":0.71899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.32600000000000001},"max":{"1min":3.8490000000000002,"5min":4.9800000000000004,"15min":6.2309999999999999},"last":0.749},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.504,"5min":3.5830000000000002,"15min":6.4279999999999999},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1031244,"kb_used_data":57540,"kb_used_omap":1,"kb_used_meta":973694,"kb_avail":93340596,"statfs":{"total":96636764160,"available":95580770304,"internally_reserved":0,"allocated":58920960,"data_stored":58755098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":997062923},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.44900000000000001},"min":{"1min":0.20100000000000001,"5min":0.19400000000000001,"15min":0.186},"max":{"1min":1.7090000000000001,"5min":3.625,"15min":10.891999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46999999999999997,"15min":0.50700000000000001},"min":{"1min":0.24399999999999999,"5min":0.219,"15min":0.19700000000000001},"max":{"1min":1.341,"5min":6.5339999999999998,"15min":9.2249999999999996},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.49199999999999999},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.20999999999999999},"max":{"1min":2.6589999999999998,"5min":13.553000000000001,"15min":13.553000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50700000000000001,"15min":0.53400000000000003},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.23300000000000001},"max":{"1min":3.008,"5min":6.6050000000000004,"15min":7.1180000000000003},"last":0.504}]},{"osd":3,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.307,"5min":0.251,"15min":0.22900000000000001},"max":{"1min":1.3120000000000001,"5min":10.236000000000001,"15min":24.879000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.53800000000000003,"15min":0.57199999999999995},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.22900000000000001},"max":{"1min":10.433,"5min":10.433,"15min":20.995000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.33300000000000002,"15min":0.30399999999999999},"max":{"1min":1.748,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.32400000000000001,"15min":0.29099999999999998},"max":{"1min":2.9390000000000001,"5min":8.4420000000000002,"15min":8.4420000000000002},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.621,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.3940000000000001,"5min":6.3940000000000001,"15min":8.6419999999999995},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80200000000000005,"15min":0.70299999999999996},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.8100000000000001,"5min":16.943999999999999,"15min":16.943999999999999},"last":0.876}]},{"osd":6,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.32100000000000001},"max":{"1min":2.379,"5min":2.379,"15min":7.468},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.33600000000000002},"max":{"1min":2.3290000000000002,"5min":3.4950000000000001,"15min":5.5030000000000001},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Mar 14 22:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.315},"max":{"1min":2.9780000000000002,"5min":6.5720000000000001,"15min":9.3209999999999997},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.36099999999999999},"max":{"1min":1.52,"5min":3.8959999999999999,"15min":10.346},"last":0.84899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":871104,"kb_used_data":49976,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":93500736,"statfs":{"total":96636764160,"available":95744753664,"internally_reserved":0,"allocated":51175424,"data_stored":51009794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":840825099},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.51700000000000002,"15min":0.46200000000000002},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.17499999999999999},"max":{"1min":0.86599999999999999,"5min":12.734,"15min":12.734},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51800000000000002,"15min":0.46400000000000002},"min":{"1min":0.25900000000000001,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":0.81000000000000005,"5min":9.3620000000000001,"15min":9.3620000000000001},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59299999999999997,"15min":0.52900000000000003},"min":{"1min":0.28399999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.96999999999999997,"5min":15.375,"15min":15.375},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.88900000000000001,"5min":9.2959999999999994,"15min":9.2959999999999994},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51500000000000001,"15min":0.46700000000000003},"min":{"1min":0.34300000000000003,"5min":0.248,"15min":0.20699999999999999},"max":{"1min":0.75,"5min":8.7230000000000008,"15min":8.7230000000000008},"last":0.432},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61299999999999999,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.90600000000000003,"5min":10.928000000000001,"15min":12.864000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71799999999999997,"15min":0.62},"min":{"1min":0.435,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.93400000000000005,"5min":9.2319999999999993,"15min":9.2319999999999993},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29699999999999999},"max":{"1min":0.92700000000000005,"5min":3.7930000000000001,"15min":7.5499999999999998},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68000000000000005,"15min":0.63200000000000001},"min":{"1min":0.39900000000000002,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":0.98499999999999999,"5min":7.8390000000000004,"15min":7.8390000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.69799999999999995},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96499999999999997,"5min":10.135999999999999,"15min":11.068},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78400000000000003,"15min":0.70999999999999996},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.002,"5min":12.77,"15min":12.77},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76400000000000001,"15min":0.66800000000000004},"min":{"1min":0.375,"5min":0.31,"15min":0.31},"max":{"1min":0.98999999999999999,"5min":9.391,"15min":9.391},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Mar 14 22:06:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.75600000000000001,"15min":0.69199999999999995},"min":{"1min":0.46400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.05,"5min":9.3290000000000006,"15min":9.3290000000000006},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.71099999999999997,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.39200000000000002,"15min":0.33300000000000002},"max":{"1min":1.014,"5min":7.8739999999999997,"15min":7.8739999999999997},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50343936,"data_stored":50332286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58679296,"data_stored":58667339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":46088192,"data_stored":46084267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":58728448,"data_stored":58720928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43065344,"data_stored":43049842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":71307264,"data_stored":71303258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":71311360,"data_stored":71303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":114360320,"data_stored":114353466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T22:06:33.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-14T22:06:33.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.909+0000 7f6d03d9f640 1 Processor -- start 2024-03-14T22:06:33.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.909+0000 7f6d03d9f640 1 -- start start 2024-03-14T22:06:33.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d03d9f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cfc0bebc0 0x7f6cfc156930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d03d9f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cfc0bb320 con 0x7f6cfc0be7a0 2024-03-14T22:06:33.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d03d9f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cfc14b760 con 0x7f6cfc06ba30 2024-03-14T22:06:33.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d03d9f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cfc0be120 con 0x7f6cfc0bebc0 2024-03-14T22:06:33.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d0133a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6cfc0be7a0 0x7f6cfc153200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46124/0 (socket says 172.21.15.49:46124) 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d0133a640 1 -- 172.21.15.49:0/2581751507 learned_addr learned my addr 172.21.15.49:0/2581751507 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d0233c640 1 --2- 172.21.15.49:0/2581751507 >> v2:172.21.15.97:3300/0 conn(0x7f6cfc0bebc0 0x7f6cfc156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038615572 0 0) 0x7f6cfc0bb320 con 0x7f6cfc0be7a0 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.910+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce8003520 con 0x7f6cfc0be7a0 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 797792547 0 0) 0x7f6ce8003520 con 0x7f6cfc0be7a0 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 >> v2:172.21.15.97:3300/0 conn(0x7f6cfc0bebc0 msgr2=0x7f6cfc156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 --2- 172.21.15.49:0/2581751507 >> v2:172.21.15.97:3300/0 conn(0x7f6cfc0bebc0 0x7f6cfc156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 >> v1:172.21.15.97:6789/0 conn(0x7f6cfc06ba30 legacy=0x7f6cfc0bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc14a9a0 con 0x7f6cfc0be7a0 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f6cec001670 con 0x7f6cfc0be7a0 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 >> v1:172.21.15.49:6789/0 conn(0x7f6cfc0be7a0 legacy=0x7f6cfc153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce8003970 0x7f6ce8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce80047c0 0x7f6ce8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ce8005180 0x7f6ce800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0be120 con 0x7f6ce80047c0 2024-03-14T22:06:33.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce800c030 con 0x7f6ce8003970 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce800c1f0 con 0x7f6ce8005180 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0133a640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce80047c0 0x7f6ce8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d01b3b640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce8003970 0x7f6ce8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0233c640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ce8005180 0x7f6ce800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0133a640 1 -- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ce8005180 msgr2=0x7f6ce800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0133a640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ce8005180 0x7f6ce800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0133a640 1 -- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce8003970 msgr2=0x7f6ce8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0133a640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce8003970 0x7f6ce8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0133a640 1 -- 172.21.15.49:0/2581751507 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce8003520 con 0x7f6ce80047c0 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d0233c640 1 -- 172.21.15.49:0/2581751507 reap_dead start 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.911+0000 7f6d01b3b640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce8003970 0x7f6ce8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.912+0000 7f6d0133a640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce80047c0 0x7f6ce8004c40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c210 tx=0x7f6ce80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1869af7e235ad0c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.912+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec005ad0 con 0x7f6ce80047c0 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.912+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec005c30 con 0x7f6ce80047c0 2024-03-14T22:06:33.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.912+0000 7f6d00b39640 1 -- 172.21.15.49:0/2581751507 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec02ca50 con 0x7f6ce80047c0 2024-03-14T22:06:33.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.913+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2581751507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce80047c0 msgr2=0x7f6ce8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.913+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce80047c0 0x7f6ce8004c40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c210 tx=0x7f6ce80057f0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.913+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2581751507 shutdown_connections 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.913+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2581751507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ce80047c0 0x7f6ce8004c40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.913+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2581751507 >> 172.21.15.49:0/2581751507 conn(0x7f6cfc0c0290 msgr2=0x7f6cfc0c06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.913+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2581751507 shutdown_connections 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.913+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2581751507 wait complete. 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 Processor -- start 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 -- start start 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80047c0 0x7f6cfc140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cfc0a4c60 0x7f6cfc1d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc1d5330 0x7f6cfc0a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc140e50 con 0x7f6cfc1d5330 2024-03-14T22:06:33.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc1e40a0 con 0x7f6ce80047c0 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.914+0000 7f6d03d9f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc1d57b0 con 0x7f6cfc0a4c60 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc1d5330 0x7f6cfc0a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc1d5330 0x7f6cfc0a3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50176/0 (socket says 172.21.15.49:50176) 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 -- 172.21.15.49:0/2163442935 learned_addr learned my addr 172.21.15.49:0/2163442935 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d01b3b640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80047c0 0x7f6cfc140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0133a640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cfc0a4c60 0x7f6cfc1d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 -- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cfc0a4c60 msgr2=0x7f6cfc1d4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cfc0a4c60 0x7f6cfc1d4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 -- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80047c0 msgr2=0x7f6cfc140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80047c0 0x7f6cfc140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 -- 172.21.15.49:0/2163442935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc1e3770 con 0x7f6cfc1d5330 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0133a640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cfc0a4c60 0x7f6cfc1d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d0233c640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc1d5330 0x7f6cfc0a3510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0a1d90 tx=0x7f6cf8011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8004030 con 0x7f6cfc1d5330 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf8004190 con 0x7f6cfc1d5330 2024-03-14T22:06:33.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cfc1e3a50 con 0x7f6cfc1d5330 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8004520 con 0x7f6cfc1d5330 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.915+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc0a50c0 con 0x7f6cfc1d5330 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.916+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cfc06c5c0 con 0x7f6cfc1d5330 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.917+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6cf800f710 con 0x7f6cfc1d5330 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.917+0000 7f6cf27fc640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6ce006a7d0 0x7f6ce006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.917+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6cf8089020 con 0x7f6cfc1d5330 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.917+0000 7f6d01b3b640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6ce006a7d0 0x7f6ce006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:33.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.918+0000 7f6d01b3b640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6ce006a7d0 0x7f6ce006cc90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000f80 tx=0x7f6cf4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:33.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:33.921+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6cf80543c0 con 0x7f6cfc1d5330 2024-03-14T22:06:34.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.101+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6cfc066bb0 con 0x7f6cfc1d5330 2024-03-14T22:06:34.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.107+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6cf80543c0 con 0x7f6cfc1d5330 2024-03-14T22:06:34.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.107+0000 7f6cf27fc640 1 -- 172.21.15.49:0/2163442935 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6cf8000e60 con 0x7f6cfc1d5330 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6ce006a7d0 msgr2=0x7f6ce006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6ce006a7d0 0x7f6ce006cc90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000f80 tx=0x7f6cf4009040 comp rx=0 tx=0).stop 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc1d5330 msgr2=0x7f6cfc0a3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc1d5330 0x7f6cfc0a3510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0a1d90 tx=0x7f6cf8011c30 comp rx=0 tx=0).stop 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 shutdown_connections 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f6ce006a7d0 0x7f6ce006cc90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6cfc1d5330 0x7f6cfc0a3510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cfc0a4c60 0x7f6cfc1d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 --2- 172.21.15.49:0/2163442935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80047c0 0x7f6cfc140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.109+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 >> 172.21.15.49:0/2163442935 conn(0x7f6cfc0c0290 msgr2=0x7f6cfc06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.110+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 shutdown_connections 2024-03-14T22:06:34.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.110+0000 7f6d03d9f640 1 -- 172.21.15.49:0/2163442935 wait complete. 2024-03-14T22:06:34.120 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T22:06:34.121 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T22:06:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.226+0000 7fec95cea640 1 Processor -- start 2024-03-14T22:06:34.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.226+0000 7fec95cea640 1 -- start start 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec95cea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 0x7fec901415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec95cea640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fec90148350 0x7fec90141b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec95cea640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec90142040 con 0x7fec9006cf10 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec95cea640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9006d670 con 0x7fec90148350 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec95cea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900b8c40 con 0x7fec900b9930 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fec90148350 0x7fec90141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fec90148350 0x7fec90141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50186/0 (socket says 172.21.15.49:50186) 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8ffff640 1 -- 172.21.15.49:0/1588285936 learned_addr learned my addr 172.21.15.49:0/1588285936 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8f7fe640 1 --1- 172.21.15.49:0/1588285936 >> v1:172.21.15.49:6789/0 conn(0x7fec9006cf10 0x7fec90147c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46130/0 (socket says 172.21.15.49:46130) 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8effd640 1 --2- 172.21.15.49:0/1588285936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 0x7fec901415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8ffff640 1 -- 172.21.15.49:0/1588285936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 msgr2=0x7fec901415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8ffff640 1 --2- 172.21.15.49:0/1588285936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 0x7fec901415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.227+0000 7fec8ffff640 1 -- 172.21.15.49:0/1588285936 >> v1:172.21.15.49:6789/0 conn(0x7fec9006cf10 legacy=0x7fec90147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.228+0000 7fec8ffff640 1 -- 172.21.15.49:0/1588285936 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec900b8f20 con 0x7fec90148350 2024-03-14T22:06:34.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.228+0000 7fec8e7fc640 1 -- 172.21.15.49:0/1588285936 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007067397 0 0) 0x7fec90142040 con 0x7fec9006cf10 2024-03-14T22:06:34.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.228+0000 7fec8ffff640 1 --2- 172.21.15.49:0/1588285936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90141b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fec8800bb10 tx=0x7fec8802e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3c688908f512a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.228+0000 7fec8e7fc640 1 -- 172.21.15.49:0/1588285936 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88036440 con 0x7fec90148350 2024-03-14T22:06:34.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.228+0000 7fec8e7fc640 1 -- 172.21.15.49:0/1588285936 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fec880365a0 con 0x7fec90148350 2024-03-14T22:06:34.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.228+0000 7fec8e7fc640 1 -- 172.21.15.49:0/1588285936 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88036950 con 0x7fec90148350 2024-03-14T22:06:34.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 -- 172.21.15.49:0/1588285936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 msgr2=0x7fec90141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 --2- 172.21.15.49:0/1588285936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90141b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fec8800bb10 tx=0x7fec8802e150 comp rx=0 tx=0).stop 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 -- 172.21.15.49:0/1588285936 shutdown_connections 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 --2- 172.21.15.49:0/1588285936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90141b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 --2- 172.21.15.49:0/1588285936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 0x7fec901415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 -- 172.21.15.49:0/1588285936 >> 172.21.15.49:0/1588285936 conn(0x7fec900b9d30 msgr2=0x7fec900ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 -- 172.21.15.49:0/1588285936 shutdown_connections 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.229+0000 7fec95cea640 1 -- 172.21.15.49:0/1588285936 wait complete. 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.230+0000 7fec95cea640 1 Processor -- start 2024-03-14T22:06:34.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.230+0000 7fec95cea640 1 -- start start 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec95cea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9006cf10 0x7fec901db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec95cea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 0x7fec901421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec95cea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec95cea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec90143680 con 0x7fec90148350 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec95cea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec901438f0 con 0x7fec9006cf10 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec95cea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900b6990 con 0x7fec900b9930 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50194/0 (socket says 172.21.15.49:50194) 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 -- 172.21.15.49:0/1676100459 learned_addr learned my addr 172.21.15.49:0/1676100459 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8f7fe640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9006cf10 0x7fec901db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 -- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 msgr2=0x7fec901421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 0x7fec901421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 -- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9006cf10 msgr2=0x7fec901db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9006cf10 0x7fec901db3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8ffff640 1 -- 172.21.15.49:0/1676100459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8800b770 con 0x7fec90148350 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.231+0000 7fec8f7fe640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9006cf10 0x7fec901db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.232+0000 7fec8ffff640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90146340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fec8800b2c0 tx=0x7fec880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.232+0000 7fec87fff640 1 -- 172.21.15.49:0/1676100459 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88036440 con 0x7fec90148350 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.232+0000 7fec87fff640 1 -- 172.21.15.49:0/1676100459 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fec88034b80 con 0x7fec90148350 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.232+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec900b6c70 con 0x7fec90148350 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.232+0000 7fec87fff640 1 -- 172.21.15.49:0/1676100459 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88041440 con 0x7fec90148350 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.232+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec901db910 con 0x7fec90148350 2024-03-14T22:06:34.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.233+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec90073540 con 0x7fec90148350 2024-03-14T22:06:34.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.233+0000 7fec87fff640 1 -- 172.21.15.49:0/1676100459 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fec8803d040 con 0x7fec90148350 2024-03-14T22:06:34.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.234+0000 7fec87fff640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fec7006a7a0 0x7fec7006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.234+0000 7fec87fff640 1 -- 172.21.15.49:0/1676100459 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fec880adba0 con 0x7fec90148350 2024-03-14T22:06:34.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.234+0000 7fec8f7fe640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fec7006a7a0 0x7fec7006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.234+0000 7fec8f7fe640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fec7006a7a0 0x7fec7006cc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fec800012e0 tx=0x7fec80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.238+0000 7fec87fff640 1 -- 172.21.15.49:0/1676100459 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fec88078ec0 con 0x7fec90148350 2024-03-14T22:06:34.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.460+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec900a1220 con 0x7fec90148350 2024-03-14T22:06:34.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.460+0000 7fec87fff640 1 -- 172.21.15.49:0/1676100459 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3969 (secure 0 0 0) 0x7fec8807c4e0 con 0x7fec90148350 2024-03-14T22:06:34.461 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:06:34.461 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2343109927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2343109927},{"type":"v1","addr":"172.21.15.97:6835","nonce":2343109927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1491968682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1491968682},{"type":"v1","addr":"172.21.15.49:6833","nonce":1491968682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T21:14:03.012761+0000","modified":"2024-03-14T21:14:05.655989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T22:06:34.461 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fec7006a7a0 msgr2=0x7fec7006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fec7006a7a0 0x7fec7006cc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fec800012e0 tx=0x7fec80009040 comp rx=0 tx=0).stop 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 msgr2=0x7fec90146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90146340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fec8800b2c0 tx=0x7fec880046c0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 shutdown_connections 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fec7006a7a0 0x7fec7006cc60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec90148350 0x7fec90146340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fec900b9930 0x7fec901421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 --2- 172.21.15.49:0/1676100459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec9006cf10 0x7fec901db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.462+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 >> 172.21.15.49:0/1676100459 conn(0x7fec900b9d30 msgr2=0x7fec9013f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.463+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 shutdown_connections 2024-03-14T22:06:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.463+0000 7fec95cea640 1 -- 172.21.15.49:0/1676100459 wait complete. 2024-03-14T22:06:34.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.579+0000 7f4e228b6640 1 Processor -- start 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.579+0000 7f4e228b6640 1 -- start start 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e228b6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4e1c0b1ba0 0x7f4e1c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e228b6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e1c0b87b0 0x7f4e1c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e228b6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1c09f340 con 0x7f4e1c0b7e10 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e228b6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c06ccf0 con 0x7f4e1c0b1ba0 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e228b6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c06d7d0 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e1b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4e1c0b1ba0 0x7f4e1c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e1bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4e1c0b7e10 0x7f4e1c0b1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46138/0 (socket says 172.21.15.49:46138) 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e1bfff640 1 -- 172.21.15.49:0/2459594570 learned_addr learned my addr 172.21.15.49:0/2459594570 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e1affd640 1 -- 172.21.15.49:0/2459594570 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503152945 0 0) 0x7f4e1c09f340 con 0x7f4e1c0b7e10 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.580+0000 7f4e20e53640 1 --2- 172.21.15.49:0/2459594570 >> v2:172.21.15.97:3300/0 conn(0x7f4e1c0b87b0 0x7f4e1c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.590+0000 7f4e1affd640 1 -- 172.21.15.49:0/2459594570 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e04007930 con 0x7f4e1c0b7e10 2024-03-14T22:06:34.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.590+0000 7f4e1b7fe640 1 -- 172.21.15.49:0/2459594570 >> v2:172.21.15.97:3300/0 conn(0x7f4e1c0b87b0 msgr2=0x7f4e1c15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:34.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.590+0000 7f4e1b7fe640 1 --2- 172.21.15.49:0/2459594570 >> v2:172.21.15.97:3300/0 conn(0x7f4e1c0b87b0 0x7f4e1c15b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.590+0000 7f4e1b7fe640 1 -- 172.21.15.49:0/2459594570 >> v1:172.21.15.49:6789/0 conn(0x7f4e1c0b7e10 legacy=0x7f4e1c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.590+0000 7f4e1affd640 1 -- 172.21.15.49:0/2459594570 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679111324 0 0) 0x7f4e04007930 con 0x7f4e1c0b7e10 2024-03-14T22:06:34.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.590+0000 7f4e1b7fe640 1 -- 172.21.15.49:0/2459594570 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c06cf70 con 0x7f4e1c0b1ba0 2024-03-14T22:06:34.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.591+0000 7f4e1b7fe640 1 --2- 172.21.15.49:0/2459594570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b1ba0 0x7f4e1c15b370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c06ef20 tx=0x7f4e10008180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f5adf01f12364 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.591+0000 7f4e1affd640 1 -- 172.21.15.49:0/2459594570 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c001670 con 0x7f4e1c0b1ba0 2024-03-14T22:06:34.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.591+0000 7f4e1affd640 1 -- 172.21.15.49:0/2459594570 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4e1000a2e0 con 0x7f4e1c0b1ba0 2024-03-14T22:06:34.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.591+0000 7f4e1affd640 1 -- 172.21.15.49:0/2459594570 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000a650 con 0x7f4e1c0b1ba0 2024-03-14T22:06:34.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 -- 172.21.15.49:0/2459594570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b1ba0 msgr2=0x7f4e1c15b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2459594570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b1ba0 0x7f4e1c15b370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c06ef20 tx=0x7f4e10008180 comp rx=0 tx=0).stop 2024-03-14T22:06:34.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 -- 172.21.15.49:0/2459594570 shutdown_connections 2024-03-14T22:06:34.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2459594570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b1ba0 0x7f4e1c15b370 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2459594570 >> v2:172.21.15.97:3300/0 conn(0x7f4e1c0b87b0 0x7f4e1c15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 -- 172.21.15.49:0/2459594570 >> 172.21.15.49:0/2459594570 conn(0x7f4e1c069ce0 msgr2=0x7f4e1c06a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:34.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 -- 172.21.15.49:0/2459594570 shutdown_connections 2024-03-14T22:06:34.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.592+0000 7f4e228b6640 1 -- 172.21.15.49:0/2459594570 wait complete. 2024-03-14T22:06:34.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.593+0000 7f4e228b6640 1 Processor -- start 2024-03-14T22:06:34.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.593+0000 7f4e228b6640 1 -- start start 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e228b6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e1c0b1ba0 0x7f4e1c154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e228b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e1c0b7e10 0x7f4e1c1ed130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e228b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b87b0 0x7f4e1c06f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e228b6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c155300 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e228b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1eca00 con 0x7f4e1c0b7e10 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e228b6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1ecca0 con 0x7f4e1c0b1ba0 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b87b0 0x7f4e1c06f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b87b0 0x7f4e1c06f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50210/0 (socket says 172.21.15.49:50210) 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 -- 172.21.15.49:0/2196432456 learned_addr learned my addr 172.21.15.49:0/2196432456 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e1b7fe640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e1c0b7e10 0x7f4e1c1ed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e1bfff640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e1c0b1ba0 0x7f4e1c154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 -- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e1c0b1ba0 msgr2=0x7f4e1c154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e1c0b1ba0 0x7f4e1c154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 -- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e1c0b7e10 msgr2=0x7f4e1c1ed130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e1c0b7e10 0x7f4e1c1ed130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 -- 172.21.15.49:0/2196432456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e10008480 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e1b7fe640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e1c0b7e10 0x7f4e1c1ed130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.594+0000 7f4e20e53640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b87b0 0x7f4e1c06f4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e14008a20 tx=0x7f4e14008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.595+0000 7f4e18ff9640 1 -- 172.21.15.49:0/2196432456 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1400edc0 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.595+0000 7f4e18ff9640 1 -- 172.21.15.49:0/2196432456 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4e14003990 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.595+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c06ef90 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.595+0000 7f4e18ff9640 1 -- 172.21.15.49:0/2196432456 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14003d60 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.595+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c1f0c70 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.596+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c1ece50 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.596+0000 7f4e18ff9640 1 -- 172.21.15.49:0/2196432456 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4e14004030 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.597+0000 7f4e18ff9640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4e0806a7d0 0x7f4e0806cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.597+0000 7f4e18ff9640 1 -- 172.21.15.49:0/2196432456 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4e14089580 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.597+0000 7f4e1bfff640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4e0806a7d0 0x7f4e0806cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.597+0000 7f4e1bfff640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4e0806a7d0 0x7f4e0806cc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c007a00 tx=0x7f4e0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.601+0000 7f4e18ff9640 1 -- 172.21.15.49:0/2196432456 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4e1c1ece50 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.821+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e1c1ece50 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.821+0000 7f4e18ff9640 1 -- 172.21.15.49:0/2196432456 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3969 (secure 0 0 0) 0x7f4e1c1ece50 con 0x7f4e1c0b87b0 2024-03-14T22:06:34.821 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:06:34.822 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2343109927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2343109927},{"type":"v1","addr":"172.21.15.97:6835","nonce":2343109927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1491968682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1491968682},{"type":"v1","addr":"172.21.15.49:6833","nonce":1491968682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T21:14:03.012761+0000","modified":"2024-03-14T21:14:05.655989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/403429269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":403429269},{"type":"v1","addr":"172.21.15.49:6835","nonce":403429269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T22:06:34.822 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-14T22:06:34.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4e0806a7d0 msgr2=0x7f4e0806cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4e0806a7d0 0x7f4e0806cc90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c007a00 tx=0x7f4e0c009040 comp rx=0 tx=0).stop 2024-03-14T22:06:34.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b87b0 msgr2=0x7f4e1c06f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b87b0 0x7f4e1c06f4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4e14008a20 tx=0x7f4e14008ee0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 shutdown_connections 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f4e0806a7d0 0x7f4e0806cc90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e1c0b87b0 0x7f4e1c06f4e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e1c0b7e10 0x7f4e1c1ed130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 --2- 172.21.15.49:0/2196432456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e1c0b1ba0 0x7f4e1c154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 >> 172.21.15.49:0/2196432456 conn(0x7f4e1c069ce0 msgr2=0x7f4e1c072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 shutdown_connections 2024-03-14T22:06:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.823+0000 7f4e228b6640 1 -- 172.21.15.49:0/2196432456 wait complete. 2024-03-14T22:06:34.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T22:06:34.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.940+0000 7fcd0ff41640 1 Processor -- start 2024-03-14T22:06:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.940+0000 7fcd0ff41640 1 -- start start 2024-03-14T22:06:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0ff41640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 0x7fcd08156e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0ff41640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0806e310 con 0x7fcd0806b1d0 2024-03-14T22:06:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0ff41640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0806d8c0 con 0x7fcd0806adf0 2024-03-14T22:06:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0ff41640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080ba5e0 con 0x7fcd08154a20 2024-03-14T22:06:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0d4dc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcd0806b1d0 0x7fcd08153300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46152/0 (socket says 172.21.15.49:46152) 2024-03-14T22:06:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/3295500223 learned_addr learned my addr 172.21.15.49:0/3295500223 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:06:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224603136 0 0) 0x7fccfc001110 con 0x7fcd0806b1d0 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccf0003520 con 0x7fcd0806b1d0 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.941+0000 7fcd0e4de640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 0x7fcd08156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1028365749 0 0) 0x7fccf0003520 con 0x7fcd0806b1d0 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 msgr2=0x7fcd08156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 0x7fcd08156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 >> v1:172.21.15.97:6789/0 conn(0x7fcd0806adf0 legacy=0x7fcd0814aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0806ce40 con 0x7fcd0806b1d0 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7fccfc001800 con 0x7fcd0806b1d0 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0e4de640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 0x7fcd08156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 >> v1:172.21.15.49:6789/0 conn(0x7fcd0806b1d0 legacy=0x7fcd08153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf00039a0 0x7fccf0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 0x7fccf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080ba5e0 con 0x7fccf00047c0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf000bb40 con 0x7fccf00039a0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf000bd00 con 0x7fcd08154a20 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0d4dc640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0e4de640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 0x7fccf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0dcdd640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf00039a0 0x7fccf0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 msgr2=0x7fccf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0d4dc640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd08154a20 0x7fccf0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf00039a0 msgr2=0x7fccf0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0d4dc640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf00039a0 0x7fccf0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.942+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/3295500223 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf0003520 con 0x7fccf00047c0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.943+0000 7fcd0d4dc640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c240 tx=0x7fccf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7329d990346f93a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.943+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc005ad0 con 0x7fccf00047c0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.943+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fccfc005c50 con 0x7fccf00047c0 2024-03-14T22:06:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.943+0000 7fcd0ccdb640 1 -- 172.21.15.49:0/3295500223 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc02c8b0 con 0x7fccf00047c0 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.944+0000 7fcd0ff41640 1 -- 172.21.15.49:0/3295500223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccf00047c0 msgr2=0x7fccf0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.944+0000 7fcd0ff41640 1 --2- 172.21.15.49:0/3295500223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccf00047c0 0x7fccf0004c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c240 tx=0x7fccf00057f0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.944+0000 7fcd0e4de640 1 -- 172.21.15.49:0/3295500223 reap_dead start 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.944+0000 7fcd0ff41640 1 -- 172.21.15.49:0/3295500223 shutdown_connections 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.944+0000 7fcd0ff41640 1 -- 172.21.15.49:0/3295500223 >> 172.21.15.49:0/3295500223 conn(0x7fcd080bad40 msgr2=0x7fcd080bb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.944+0000 7fcd0ff41640 1 -- 172.21.15.49:0/3295500223 shutdown_connections 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.944+0000 7fcd0ff41640 1 -- 172.21.15.49:0/3295500223 wait complete. 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.945+0000 7fcd0ff41640 1 Processor -- start 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.945+0000 7fcd0ff41640 1 -- start start 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.945+0000 7fcd0ff41640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fccf00047c0 0x7fcd081e3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.945+0000 7fcd0ff41640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd081454e0 0x7fcd08145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.945+0000 7fcd0ff41640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08140960 0x7fcd08140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.945+0000 7fcd0ff41640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081e37b0 con 0x7fcd081454e0 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.945+0000 7fcd0ff41640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd08145ea0 con 0x7fcd08140960 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0ff41640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080a3ee0 con 0x7fccf00047c0 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd081454e0 0x7fcd08145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd081454e0 0x7fcd08145960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50228/0 (socket says 172.21.15.49:50228) 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/606681752 learned_addr learned my addr 172.21.15.49:0/606681752 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0e4de640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08140960 0x7fcd08140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0dcdd640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fccf00047c0 0x7fcd081e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fccf00047c0 msgr2=0x7fcd081e3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fccf00047c0 0x7fcd081e3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08140960 msgr2=0x7fcd08140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08140960 0x7fcd08140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 -- 172.21.15.49:0/606681752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080a27a0 con 0x7fcd081454e0 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0e4de640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08140960 0x7fcd08140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T22:06:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0d4dc640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd081454e0 0x7fcd08145960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fccfc02c450 tx=0x7fccfc001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd067fc640 1 -- 172.21.15.49:0/606681752 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc039070 con 0x7fcd081454e0 2024-03-14T22:06:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd067fc640 1 -- 172.21.15.49:0/606681752 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fccfc02cab0 con 0x7fcd081454e0 2024-03-14T22:06:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd067fc640 1 -- 172.21.15.49:0/606681752 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc03c040 con 0x7fcd081454e0 2024-03-14T22:06:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd080a2a80 con 0x7fcd081454e0 2024-03-14T22:06:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.946+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd080a31e0 con 0x7fcd081454e0 2024-03-14T22:06:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.947+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd08073540 con 0x7fcd081454e0 2024-03-14T22:06:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.948+0000 7fcd067fc640 1 -- 172.21.15.49:0/606681752 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fccfc032070 con 0x7fcd081454e0 2024-03-14T22:06:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.948+0000 7fcd067fc640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fccf406a7a0 0x7fccf406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.948+0000 7fcd067fc640 1 -- 172.21.15.49:0/606681752 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fccfc0aa310 con 0x7fcd081454e0 2024-03-14T22:06:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.948+0000 7fcd0dcdd640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fccf406a7a0 0x7fccf406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.949+0000 7fcd0dcdd640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fccf406a7a0 0x7fccf406cc60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd000035f0 tx=0x7fcd00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:34.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:34.952+0000 7fcd067fc640 1 -- 172.21.15.49:0/606681752 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fccfc0756b0 con 0x7fcd081454e0 2024-03-14T22:06:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.134+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd0806e310 con 0x7fcd081454e0 2024-03-14T22:06:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.135+0000 7fcd067fc640 1 -- 172.21.15.49:0/606681752 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16239 (secure 0 0 0) 0x7fcd0806e310 con 0x7fcd081454e0 2024-03-14T22:06:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-14T22:06:35.135 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"25282add-1360-4a83-a2da-1a7e2b367f69","created":"2024-03-14T21:13:38.767442+0000","modified":"2024-03-14T21:14:44.734113+0000","last_up_change":"2024-03-14T21:13:49.594913+0000","last_in_change":"2024-03-14T21:13:40.926498+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T21:13:50.964908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T21:14:01.020268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T21:14:01.995660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d49ebb7b-cd2e-4498-9607-a6bbdcefdb51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":37908},{"type":"v1","addr":"172.21.15.49:6817","nonce":37908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":37908},{"type":"v1","addr":"172.21.15.49:6819","nonce":37908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":37908},{"type":"v1","addr":"172.21.15.49:6823","nonce":37908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":37908},{"type":"v1","addr":"172.21.15.49:6821","nonce":37908}]},"public_addr":"172.21.15.49:6817/37908","cluster_addr":"172.21.15.49:6819/37908","heartbeat_back_addr":"172.21.15.49:6823/37908","heartbeat_front_addr":"172.21.15.49:6821/37908","state":["exists","up"]},{"osd":1,"uuid":"07801ea6-7081-49cf-ab8b-cfcb4a9c8113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":37907},{"type":"v1","addr":"172.21.15.49:6801","nonce":37907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":37907},{"type":"v1","addr":"172.21.15.49:6803","nonce":37907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":37907},{"type":"v1","addr":"172.21.15.49:6807","nonce":37907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":37907},{"type":"v1","addr":"172.21.15.49:6805","nonce":37907}]},"public_addr":"172.21.15.49:6801/37907","cluster_addr":"172.21.15.49:6803/37907","heartbeat_back_addr":"172.21.15.49:6807/37907","heartbeat_front_addr":"172.21.15.49:6805/37907","state":["exists","up"]},{"osd":2,"uuid":"698263f6-bc7c-41a8-aa99-be46814ddb8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":37918},{"type":"v1","addr":"172.21.15.49:6825","nonce":37918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":37918},{"type":"v1","addr":"172.21.15.49:6827","nonce":37918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":37918},{"type":"v1","addr":"172.21.15.49:6831","nonce":37918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":37918},{"type":"v1","addr":"172.21.15.49:6829","nonce":37918}]},"public_addr":"172.21.15.49:6825/37918","cluster_addr":"172.21.15.49:6827/37918","heartbeat_back_addr":"172.21.15.49:6831/37918","heartbeat_front_addr":"172.21.15.49:6829/37918","state":["exists","up"]},{"osd":3,"uuid":"c551f38b-c910-4d22-b9de-5e8fc2356379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":37909},{"type":"v1","addr":"172.21.15.49:6809","nonce":37909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":37909},{"type":"v1","addr":"172.21.15.49:6811","nonce":37909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":37909},{"type":"v1","addr":"172.21.15.49:6815","nonce":37909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":37909},{"type":"v1","addr":"172.21.15.49:6813","nonce":37909}]},"public_addr":"172.21.15.49:6809/37909","cluster_addr":"172.21.15.49:6811/37909","heartbeat_back_addr":"172.21.15.49:6815/37909","heartbeat_front_addr":"172.21.15.49:6813/37909","state":["exists","up"]},{"osd":4,"uuid":"cd43c8e4-a723-4ed4-8c61-dea1188c1b35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":37904},{"type":"v1","addr":"172.21.15.97:6819","nonce":37904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":37904},{"type":"v1","addr":"172.21.15.97:6823","nonce":37904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":37904},{"type":"v1","addr":"172.21.15.97:6831","nonce":37904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":37904},{"type":"v1","addr":"172.21.15.97:6827","nonce":37904}]},"public_addr":"172.21.15.97:6819/37904","cluster_addr":"172.21.15.97:6823/37904","heartbeat_back_addr":"172.21.15.97:6831/37904","heartbeat_front_addr":"172.21.15.97:6827/37904","state":["exists","up"]},{"osd":5,"uuid":"d527b87c-b6bd-4a30-9c49-a0e640234a12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":37921},{"type":"v1","addr":"172.21.15.97:6818","nonce":37921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":37921},{"type":"v1","addr":"172.21.15.97:6822","nonce":37921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":37921},{"type":"v1","addr":"172.21.15.97:6830","nonce":37921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":37921},{"type":"v1","addr":"172.21.15.97:6826","nonce":37921}]},"public_addr":"172.21.15.97:6818/37921","cluster_addr":"172.21.15.97:6822/37921","heartbeat_back_addr":"172.21.15.97:6830/37921","heartbeat_front_addr":"172.21.15.97:6826/37921","state":["exists","up"]},{"osd":6,"uuid":"17236019-5f1b-4e4f-b3ab-bb0b664551b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":37902},{"type":"v1","addr":"172.21.15.97:6801","nonce":37902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":37902},{"type":"v1","addr":"172.21.15.97:6803","nonce":37902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":37902},{"type":"v1","addr":"172.21.15.97:6807","nonce":37902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":37902},{"type":"v1","addr":"172.21.15.97:6805","nonce":37902}]},"public_addr":"172.21.15.97:6801/37902","cluster_addr":"172.21.15.97:6803/37902","heartbeat_back_addr":"172.21.15.97:6807/37902","heartbeat_front_addr":"172.21.15.97:6805/37902","state":["exists","up"]},{"osd":7,"uuid":"e9bae753-4d72-434b-8919-02054a33b4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":37903},{"type":"v1","addr":"172.21.15.97:6809","nonce":37903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":37903},{"type":"v1","addr":"172.21.15.97:6811","nonce":37903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":37903},{"type":"v1","addr":"172.21.15.97:6815","nonce":37903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":37903},{"type":"v1","addr":"172.21.15.97:6813","nonce":37903}]},"public_addr":"172.21.15.97:6809/37903","cluster_addr":"172.21.15.97:6811/37903","heartbeat_back_addr":"172.21.15.97:6815/37903","heartbeat_front_addr":"172.21.15.97:6813/37903","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.142516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.087763+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.161694+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.141864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.083681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.065264+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T21:13:46.034321+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T22:06:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fccf406a7a0 msgr2=0x7fccf406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fccf406a7a0 0x7fccf406cc60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd000035f0 tx=0x7fcd00009040 comp rx=0 tx=0).stop 2024-03-14T22:06:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd081454e0 msgr2=0x7fcd08145960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd081454e0 0x7fcd08145960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fccfc02c450 tx=0x7fccfc001b90 comp rx=0 tx=0).stop 2024-03-14T22:06:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 shutdown_connections 2024-03-14T22:06:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fccf406a7a0 0x7fccf406cc60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08140960 0x7fcd08140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd081454e0 0x7fcd08145960 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 --2- 172.21.15.49:0/606681752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fccf00047c0 0x7fcd081e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 >> 172.21.15.49:0/606681752 conn(0x7fcd080bad40 msgr2=0x7fcd080bc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 shutdown_connections 2024-03-14T22:06:35.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.137+0000 7fcd0ff41640 1 -- 172.21.15.49:0/606681752 wait complete. 2024-03-14T22:06:35.148 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T22:06:35.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.257+0000 7f248b0d5640 1 Processor -- start 2024-03-14T22:06:35.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.257+0000 7f248b0d5640 1 -- start start 2024-03-14T22:06:35.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.257+0000 7f248b0d5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f248414b820 0x7f248406a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.257+0000 7f248b0d5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24840bb1c0 0x7f248414e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.257+0000 7f248b0d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24840b3a20 con 0x7f24840bad60 2024-03-14T22:06:35.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.257+0000 7f248b0d5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24840ba550 con 0x7f248414b820 2024-03-14T22:06:35.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.257+0000 7f248b0d5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840bfbe0 con 0x7f24840bb1c0 2024-03-14T22:06:35.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2488e71640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f248414b820 0x7f248406a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2489672640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24840bb1c0 0x7f248414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2483fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f24840bad60 0x7f248414e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:38454/0 (socket says 172.21.15.49:38454) 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2483fff640 1 -- 172.21.15.49:0/3034137138 learned_addr learned my addr 172.21.15.49:0/3034137138 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2488e71640 1 -- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24840bb1c0 msgr2=0x7f248414e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2488e71640 1 --2- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24840bb1c0 0x7f248414e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2488e71640 1 -- 172.21.15.49:0/3034137138 >> v1:172.21.15.97:6789/0 conn(0x7f24840bad60 legacy=0x7f248414e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f24837fe640 1 -- 172.21.15.49:0/3034137138 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985440027 0 0) 0x7f24840b3a20 con 0x7f24840bad60 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2488e71640 1 -- 172.21.15.49:0/3034137138 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24840bfec0 con 0x7f248414b820 2024-03-14T22:06:35.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.258+0000 7f2489672640 1 --2- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24840bb1c0 0x7f248414e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.259+0000 7f2488e71640 1 --2- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248414b820 0x7f248406a6e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24780097a0 tx=0x7f247802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64be6f64920e5022 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.259+0000 7f24837fe640 1 -- 172.21.15.49:0/3034137138 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478031830 con 0x7f248414b820 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.259+0000 7f24837fe640 1 -- 172.21.15.49:0/3034137138 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2478031990 con 0x7f248414b820 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.259+0000 7f24837fe640 1 -- 172.21.15.49:0/3034137138 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478031d40 con 0x7f248414b820 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 -- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248414b820 msgr2=0x7f248406a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 --2- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248414b820 0x7f248406a6e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24780097a0 tx=0x7f247802b8d0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 -- 172.21.15.49:0/3034137138 shutdown_connections 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 --2- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24840bb1c0 0x7f248414e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 --2- 172.21.15.49:0/3034137138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248414b820 0x7f248406a6e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 -- 172.21.15.49:0/3034137138 >> 172.21.15.49:0/3034137138 conn(0x7f24840bcd80 msgr2=0x7f24840bd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 -- 172.21.15.49:0/3034137138 shutdown_connections 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.260+0000 7f248b0d5640 1 -- 172.21.15.49:0/3034137138 wait complete. 2024-03-14T22:06:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.261+0000 7f248b0d5640 1 Processor -- start 2024-03-14T22:06:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.261+0000 7f248b0d5640 1 -- start start 2024-03-14T22:06:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f248b0d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24840bad60 0x7f24840b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f248b0d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24840bb1c0 0x7f24840b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f248b0d5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f248414b820 0x7f24840b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f248b0d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248409a0c0 con 0x7f24840bb1c0 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f248b0d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840b63f0 con 0x7f24840bad60 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f248b0d5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248409a2a0 con 0x7f248414b820 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24840bb1c0 0x7f24840b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24840bb1c0 0x7f24840b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50230/0 (socket says 172.21.15.49:50230) 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 -- 172.21.15.49:0/2322473578 learned_addr learned my addr 172.21.15.49:0/2322473578 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2488e71640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24840bad60 0x7f24840b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2489672640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f248414b820 0x7f24840b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 -- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f248414b820 msgr2=0x7f24840b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f248414b820 0x7f24840b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 -- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24840bad60 msgr2=0x7f24840b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24840bad60 0x7f24840b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 -- 172.21.15.49:0/2322473578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c009a70 con 0x7f24840bb1c0 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f2483fff640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24840bb1c0 0x7f24840b47c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f247000c500 tx=0x7f247000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f24817fa640 1 -- 172.21.15.49:0/2322473578 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470007560 con 0x7f24840bb1c0 2024-03-14T22:06:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f24817fa640 1 -- 172.21.15.49:0/2322473578 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f24700047e0 con 0x7f24840bb1c0 2024-03-14T22:06:35.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f24817fa640 1 -- 172.21.15.49:0/2322473578 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470004bb0 con 0x7f24840bb1c0 2024-03-14T22:06:35.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.262+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2478009400 con 0x7f24840bb1c0 2024-03-14T22:06:35.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.263+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24841dff80 con 0x7f24840bb1c0 2024-03-14T22:06:35.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.264+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484066bb0 con 0x7f24840bb1c0 2024-03-14T22:06:35.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.264+0000 7f24817fa640 1 -- 172.21.15.49:0/2322473578 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2470004d10 con 0x7f24840bb1c0 2024-03-14T22:06:35.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.264+0000 7f24817fa640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f246806a7a0 0x7f246806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.264+0000 7f24817fa640 1 -- 172.21.15.49:0/2322473578 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2470087fb0 con 0x7f24840bb1c0 2024-03-14T22:06:35.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.265+0000 7f2488e71640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f246806a7a0 0x7f246806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.265+0000 7f2488e71640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f246806a7a0 0x7f246806cc60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2478036000 tx=0x7f247803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:35.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.269+0000 7f24817fa640 1 -- 172.21.15.49:0/2322473578 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2470053350 con 0x7f24840bb1c0 2024-03-14T22:06:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.488+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f248406b470 con 0x7f24840bb1c0 2024-03-14T22:06:35.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.527+0000 7f24817fa640 1 -- 172.21.15.49:0/2322473578 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2470018090 con 0x7f24840bb1c0 2024-03-14T22:06:35.528 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f246806a7a0 msgr2=0x7f246806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f246806a7a0 0x7f246806cc60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2478036000 tx=0x7f247803c040 comp rx=0 tx=0).stop 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24840bb1c0 msgr2=0x7f24840b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24840bb1c0 0x7f24840b47c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f247000c500 tx=0x7f247000c9c0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 shutdown_connections 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f246806a7a0 0x7f246806cc60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f248414b820 0x7f24840b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24840bb1c0 0x7f24840b47c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 --2- 172.21.15.49:0/2322473578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24840bad60 0x7f24840b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 >> 172.21.15.49:0/2322473578 conn(0x7f24840bcd80 msgr2=0x7f24840a0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:35.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 shutdown_connections 2024-03-14T22:06:35.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.530+0000 7f248b0d5640 1 -- 172.21.15.49:0/2322473578 wait complete. 2024-03-14T22:06:35.541 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T22:06:35.599 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-03-14T22:06:35.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.651+0000 7fcd581e7640 1 Processor -- start 2024-03-14T22:06:35.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.651+0000 7fcd581e7640 1 -- start start 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd581e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd500bae20 0x7fcd5014df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd581e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 0x7fcd501552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd581e7640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500be810 con 0x7fcd501449c0 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd581e7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500bc550 con 0x7fcd500bae20 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd581e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5014e490 con 0x7fcd500bf3b0 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd55782640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcd501449c0 0x7fcd500beca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46168/0 (socket says 172.21.15.49:46168) 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd55782640 1 -- 172.21.15.49:0/2001456483 learned_addr learned my addr 172.21.15.49:0/2001456483 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd56784640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 0x7fcd501552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200369122 0 0) 0x7fcd500be810 con 0x7fcd501449c0 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd55f83640 1 --2- 172.21.15.49:0/2001456483 >> v2:172.21.15.97:3300/0 conn(0x7fcd500bae20 0x7fcd5014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.652+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd3c003520 con 0x7fcd501449c0 2024-03-14T22:06:35.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472534114 0 0) 0x7fcd3c003520 con 0x7fcd501449c0 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 msgr2=0x7fcd501552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 0x7fcd501552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 >> v2:172.21.15.97:3300/0 conn(0x7fcd500bae20 msgr2=0x7fcd5014df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 --2- 172.21.15.49:0/2001456483 >> v2:172.21.15.97:3300/0 conn(0x7fcd500bae20 0x7fcd5014df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5014e700 con 0x7fcd501449c0 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7fcd40001670 con 0x7fcd501449c0 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 >> v1:172.21.15.49:6789/0 conn(0x7fcd501449c0 legacy=0x7fcd500beca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 0x7fcd3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd3c004d00 0x7fcd3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500bc550 con 0x7fcd3c004d00 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5014e490 con 0x7fcd3c003970 2024-03-14T22:06:35.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c00b980 con 0x7fcd500bf3b0 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd56784640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd3c004d00 0x7fcd3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd55782640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 0x7fcd3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd55f83640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd56784640 1 -- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 msgr2=0x7fcd3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd56784640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd500bf3b0 0x7fcd3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd56784640 1 -- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd3c003970 msgr2=0x7fcd3c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd56784640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.653+0000 7fcd56784640 1 -- 172.21.15.49:0/2001456483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c003520 con 0x7fcd3c004d00 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.654+0000 7fcd56784640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd3c004d00 0x7fcd3c005180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009fc0 tx=0x7fcd3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf54a1321cf6418c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.654+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4802cc20 con 0x7fcd3c004d00 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.654+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcd4800f040 con 0x7fcd3c004d00 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.654+0000 7fcd54f81640 1 -- 172.21.15.49:0/2001456483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48034440 con 0x7fcd3c004d00 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.655+0000 7fcd581e7640 1 -- 172.21.15.49:0/2001456483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd3c004d00 msgr2=0x7fcd3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.655+0000 7fcd581e7640 1 --2- 172.21.15.49:0/2001456483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd3c004d00 0x7fcd3c005180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009fc0 tx=0x7fcd3c0057f0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.655+0000 7fcd56784640 1 -- 172.21.15.49:0/2001456483 reap_dead start 2024-03-14T22:06:35.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.655+0000 7fcd581e7640 1 -- 172.21.15.49:0/2001456483 shutdown_connections 2024-03-14T22:06:35.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.655+0000 7fcd581e7640 1 -- 172.21.15.49:0/2001456483 >> 172.21.15.49:0/2001456483 conn(0x7fcd500c0230 msgr2=0x7fcd500c0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:35.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.655+0000 7fcd581e7640 1 -- 172.21.15.49:0/2001456483 shutdown_connections 2024-03-14T22:06:35.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.655+0000 7fcd581e7640 1 -- 172.21.15.49:0/2001456483 wait complete. 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 Processor -- start 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 -- start start 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd3c004d00 0x7fcd50148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd50149b30 0x7fcd501439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd500a5350 0x7fcd5014a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501e37a0 con 0x7fcd50149b30 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50149f70 con 0x7fcd500a5350 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd581e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501e3a10 con 0x7fcd3c004d00 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd55782640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd50149b30 0x7fcd501439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd55782640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd50149b30 0x7fcd501439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50266/0 (socket says 172.21.15.49:50266) 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.656+0000 7fcd55782640 1 -- 172.21.15.49:0/3425750529 learned_addr learned my addr 172.21.15.49:0/3425750529 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:35.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd56784640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd500a5350 0x7fcd5014a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd55782640 1 -- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd3c004d00 msgr2=0x7fcd50148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd55782640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd3c004d00 0x7fcd50148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd55782640 1 -- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd500a5350 msgr2=0x7fcd5014a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd55782640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd500a5350 0x7fcd5014a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd55782640 1 -- 172.21.15.49:0/3425750529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd501e3cf0 con 0x7fcd50149b30 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd56784640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd500a5350 0x7fcd5014a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd55782640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd50149b30 0x7fcd501439b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd40007310 tx=0x7fcd400077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd467fc640 1 -- 172.21.15.49:0/3425750529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000f9a0 con 0x7fcd50149b30 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd467fc640 1 -- 172.21.15.49:0/3425750529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcd4000fb00 con 0x7fcd50149b30 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd501e3fd0 con 0x7fcd50149b30 2024-03-14T22:06:35.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd467fc640 1 -- 172.21.15.49:0/3425750529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000fed0 con 0x7fcd50149b30 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.657+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd500a5db0 con 0x7fcd50149b30 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.658+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd50066bb0 con 0x7fcd50149b30 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.659+0000 7fcd467fc640 1 -- 172.21.15.49:0/3425750529 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcd4001d460 con 0x7fcd50149b30 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.659+0000 7fcd467fc640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd3406a7a0 0x7fcd3406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.659+0000 7fcd467fc640 1 -- 172.21.15.49:0/3425750529 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fcd400891d0 con 0x7fcd50149b30 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.659+0000 7fcd55f83640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd3406a7a0 0x7fcd3406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.660+0000 7fcd55f83640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd3406a7a0 0x7fcd3406cc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006b10 tx=0x7fcd4c006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:35.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.663+0000 7fcd467fc640 1 -- 172.21.15.49:0/3425750529 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcd40054520 con 0x7fcd50149b30 2024-03-14T22:06:35.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:35.882+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcd5006b3c0 con 0x7fcd50149b30 2024-03-14T22:06:36.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.531+0000 7fcd467fc640 1 -- 172.21.15.49:0/3425750529 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcd40057b40 con 0x7fcd50149b30 2024-03-14T22:06:36.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd3406a7a0 msgr2=0x7fcd3406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd3406a7a0 0x7fcd3406cc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006b10 tx=0x7fcd4c006aa0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd50149b30 msgr2=0x7fcd501439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd50149b30 0x7fcd501439b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd40007310 tx=0x7fcd400077d0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 shutdown_connections 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fcd3406a7a0 0x7fcd3406cc60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd500a5350 0x7fcd5014a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd50149b30 0x7fcd501439b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 --2- 172.21.15.49:0/3425750529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd3c004d00 0x7fcd50148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 >> 172.21.15.49:0/3425750529 conn(0x7fcd500c0230 msgr2=0x7fcd5014be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 shutdown_connections 2024-03-14T22:06:36.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.534+0000 7fcd581e7640 1 -- 172.21.15.49:0/3425750529 wait complete. 2024-03-14T22:06:36.547 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T22:06:36.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.660+0000 7f37cf096640 1 Processor -- start 2024-03-14T22:06:36.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.661+0000 7f37cf096640 1 -- start start 2024-03-14T22:06:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.661+0000 7f37cf096640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37c80bfc80 0x7f37c814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.661+0000 7f37cf096640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c806ae40 con 0x7f37c80bb010 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.661+0000 7f37cf096640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c80bc590 con 0x7f37c80bc940 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.661+0000 7f37cf096640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c80ba750 con 0x7f37c80bfc80 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.661+0000 7f37cd633640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f37c80bb010 0x7f37c81555c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46174/0 (socket says 172.21.15.49:46174) 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.661+0000 7f37cd633640 1 -- 172.21.15.49:0/3614879902 learned_addr learned my addr 172.21.15.49:0/3614879902 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37cce32640 1 --2- 172.21.15.49:0/3614879902 >> v2:172.21.15.97:3300/0 conn(0x7f37c80bfc80 0x7f37c814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820074677 0 0) 0x7f37c806ae40 con 0x7f37c80bb010 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b8003520 con 0x7f37c80bb010 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215128399 0 0) 0x7f37c80bc590 con 0x7f37c80bc940 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c806ae40 con 0x7f37c80bc940 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3077134558 0 0) 0x7f37b8003520 con 0x7f37c80bb010 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 >> v2:172.21.15.97:3300/0 conn(0x7f37c80bfc80 msgr2=0x7f37c814ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 --2- 172.21.15.49:0/3614879902 >> v2:172.21.15.97:3300/0 conn(0x7f37c80bfc80 0x7f37c814ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 >> v1:172.21.15.97:6789/0 conn(0x7f37c80bc940 legacy=0x7f37c80b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c806a880 con 0x7f37c80bb010 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 117181214 0 0) 0x7f37c00019f0 con 0x7f37c80bb010 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 >> v1:172.21.15.49:6789/0 conn(0x7f37c80bb010 legacy=0x7f37c81555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37b8003be0 0x7f37b8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37b80047c0 0x7f37b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b8005350 0x7f37b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80ba750 con 0x7f37b80047c0 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b800c220 con 0x7f37b8005350 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b800c3e0 con 0x7f37b8003be0 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c7fff640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37b80047c0 0x7f37b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37cce32640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37b8003be0 0x7f37b8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37cd633640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b8005350 0x7f37b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c7fff640 1 -- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37b8003be0 msgr2=0x7f37b8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:36.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c7fff640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37b8003be0 0x7f37b8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c7fff640 1 -- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b8005350 msgr2=0x7f37b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c7fff640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b8005350 0x7f37b800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37c7fff640 1 -- 172.21.15.49:0/3614879902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8003520 con 0x7f37b80047c0 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.662+0000 7f37cd633640 1 -- 172.21.15.49:0/3614879902 reap_dead start 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.663+0000 7f37c7fff640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37b80047c0 0x7f37b8004c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37b40018a0 tx=0x7f37b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eebaa0cc825994 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.663+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40052d0 con 0x7f37b80047c0 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.663+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f37b4005430 con 0x7f37b80047c0 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.663+0000 7f37c77fe640 1 -- 172.21.15.49:0/3614879902 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4003830 con 0x7f37b80047c0 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.664+0000 7f37cf096640 1 -- 172.21.15.49:0/3614879902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37b80047c0 msgr2=0x7f37b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.664+0000 7f37cf096640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37b80047c0 0x7f37b8004c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37b40018a0 tx=0x7f37b80057f0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.664+0000 7f37cf096640 1 -- 172.21.15.49:0/3614879902 shutdown_connections 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.664+0000 7f37cf096640 1 --2- 172.21.15.49:0/3614879902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37b80047c0 0x7f37b8004c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.664+0000 7f37cf096640 1 -- 172.21.15.49:0/3614879902 >> 172.21.15.49:0/3614879902 conn(0x7f37c80bcda0 msgr2=0x7f37c80bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.664+0000 7f37cf096640 1 -- 172.21.15.49:0/3614879902 shutdown_connections 2024-03-14T22:06:36.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.664+0000 7f37cf096640 1 -- 172.21.15.49:0/3614879902 wait complete. 2024-03-14T22:06:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.665+0000 7f37cf096640 1 Processor -- start 2024-03-14T22:06:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.665+0000 7f37cf096640 1 -- start start 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cf096640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b80047c0 0x7f37c814e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cf096640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37c8147700 0x7f37c81e3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cf096640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37c81e3b80 0x7f37c8146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cf096640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c814ed60 con 0x7f37c8147700 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cf096640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80a50a0 con 0x7f37b80047c0 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cf096640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80b6bc0 con 0x7f37c81e3b80 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37c8147700 0x7f37c81e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37c8147700 0x7f37c81e3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50276/0 (socket says 172.21.15.49:50276) 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 -- 172.21.15.49:0/1675561600 learned_addr learned my addr 172.21.15.49:0/1675561600 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cce32640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b80047c0 0x7f37c814e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37cd633640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37c81e3b80 0x7f37c8146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 -- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37c81e3b80 msgr2=0x7f37c8146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37c81e3b80 0x7f37c8146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 -- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b80047c0 msgr2=0x7f37c814e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b80047c0 0x7f37c814e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 -- 172.21.15.49:0/1675561600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80b6640 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c7fff640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37c8147700 0x7f37c81e3640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37b40018a0 tx=0x7f37b40012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.666+0000 7f37c57fa640 1 -- 172.21.15.49:0/1675561600 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4007250 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.667+0000 7f37c57fa640 1 -- 172.21.15.49:0/1675561600 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f37b40073b0 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.667+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c80b68e0 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.667+0000 7f37c57fa640 1 -- 172.21.15.49:0/1675561600 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40035f0 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.667+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c80b6f80 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.668+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c8066bb0 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.668+0000 7f37c57fa640 1 -- 172.21.15.49:0/1675561600 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f37b4003750 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.669+0000 7f37c57fa640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f37ac06a7d0 0x7f37ac06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.669+0000 7f37c57fa640 1 -- 172.21.15.49:0/1675561600 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f37b40aa470 con 0x7f37c8147700 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.669+0000 7f37cce32640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f37ac06a7d0 0x7f37ac06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.669+0000 7f37cce32640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f37ac06a7d0 0x7f37ac06cc90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009b60 tx=0x7f37bc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:36.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.673+0000 7f37c57fa640 1 -- 172.21.15.49:0/1675561600 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f37b4075930 con 0x7f37c8147700 2024-03-14T22:06:36.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:36.851+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37c806b490 con 0x7f37c8147700 2024-03-14T22:06:37.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.535+0000 7f37c57fa640 1 -- 172.21.15.49:0/1675561600 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f37b4078f50 con 0x7f37c8147700 2024-03-14T22:06:37.535 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-03-14T22:06:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.538+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f37ac06a7d0 msgr2=0x7f37ac06cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.538+0000 7f37cf096640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f37ac06a7d0 0x7f37ac06cc90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009b60 tx=0x7f37bc009190 comp rx=0 tx=0).stop 2024-03-14T22:06:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37c8147700 msgr2=0x7f37c81e3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37c8147700 0x7f37c81e3640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37b40018a0 tx=0x7f37b40012d0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 shutdown_connections 2024-03-14T22:06:37.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7f37ac06a7d0 0x7f37ac06cc90 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009b60 tx=0x7f37bc009190 comp rx=0 tx=0).stop 2024-03-14T22:06:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f37c81e3b80 0x7f37c8146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37c8147700 0x7f37c81e3640 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 --2- 172.21.15.49:0/1675561600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b80047c0 0x7f37c814e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 >> 172.21.15.49:0/1675561600 conn(0x7f37c80bcda0 msgr2=0x7f37c80bd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 shutdown_connections 2024-03-14T22:06:37.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.539+0000 7f37cf096640 1 -- 172.21.15.49:0/1675561600 wait complete. 2024-03-14T22:06:37.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T22:06:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.699+0000 7fe6b4026640 1 Processor -- start 2024-03-14T22:06:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.700+0000 7fe6b4026640 1 -- start start 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.700+0000 7fe6b4026640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6ac0bfc80 0x7fe6ac14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.700+0000 7fe6b4026640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac0bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.700+0000 7fe6b4026640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ac06ae40 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.700+0000 7fe6b4026640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ac0ba750 con 0x7fe6ac0bfc80 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.700+0000 7fe6b4026640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac0bb550 con 0x7fe6ac0bc940 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b15c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac0bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b1dc2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6ac0bfc80 0x7fe6ac14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b25c3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe6ac0b9c00 0x7fe6ac14e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.49:38462/0 (socket says 172.21.15.49:38462) 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b1dc2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6ac0bfc80 0x7fe6ac14ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.49:54338/0 (socket says 172.21.15.49:54338) 2024-03-14T22:06:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b25c3640 1 -- 172.21.15.49:0/3341734574 learned_addr learned my addr 172.21.15.49:0/3341734574 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-03-14T22:06:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b15c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac0bb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.49:36438/0 (socket says 172.21.15.49:36438) 2024-03-14T22:06:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b15c1640 1 -- 172.21.15.49:0/3341734574 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac0bfc80 msgr2=0x7fe6ac14ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T22:06:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b15c1640 1 --2- 172.21.15.49:0/3341734574 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac0bfc80 0x7fe6ac14ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b15c1640 1 -- 172.21.15.49:0/3341734574 >> v1:172.21.15.97:6789/0 conn(0x7fe6ac0b9c00 legacy=0x7fe6ac14e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b0dc0640 1 -- 172.21.15.49:0/3341734574 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898268934 0 0) 0x7fe6ac06ae40 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b15c1640 1 -- 172.21.15.49:0/3341734574 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac0bc590 con 0x7fe6ac0bc940 2024-03-14T22:06:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.701+0000 7fe6b1dc2640 1 --2- 172.21.15.49:0/3341734574 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac0bfc80 0x7fe6ac14ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T22:06:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b15c1640 1 --2- 172.21.15.49:0/3341734574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac0bb010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097a0 tx=0x7fe69c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373770555482f527 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b0dc0640 1 -- 172.21.15.49:0/3341734574 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0316f0 con 0x7fe6ac0bc940 2024-03-14T22:06:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b0dc0640 1 -- 172.21.15.49:0/3341734574 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe69c031850 con 0x7fe6ac0bc940 2024-03-14T22:06:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b0dc0640 1 -- 172.21.15.49:0/3341734574 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c031c00 con 0x7fe6ac0bc940 2024-03-14T22:06:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b4026640 1 -- 172.21.15.49:0/3341734574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 msgr2=0x7fe6ac0bb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b4026640 1 --2- 172.21.15.49:0/3341734574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac0bb010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097a0 tx=0x7fe69c02b790 comp rx=0 tx=0).stop 2024-03-14T22:06:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b4026640 1 -- 172.21.15.49:0/3341734574 shutdown_connections 2024-03-14T22:06:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b4026640 1 --2- 172.21.15.49:0/3341734574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac0bb010 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b4026640 1 --2- 172.21.15.49:0/3341734574 >> v2:172.21.15.97:3300/0 conn(0x7fe6ac0bfc80 0x7fe6ac14ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.702+0000 7fe6b4026640 1 -- 172.21.15.49:0/3341734574 >> 172.21.15.49:0/3341734574 conn(0x7fe6ac0bcda0 msgr2=0x7fe6ac0bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.703+0000 7fe6b4026640 1 -- 172.21.15.49:0/3341734574 shutdown_connections 2024-03-14T22:06:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.703+0000 7fe6b4026640 1 -- 172.21.15.49:0/3341734574 wait complete. 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 Processor -- start 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 -- start start 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6ac0b9c00 0x7fe6ac0b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac1e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0bfc80 0x7fe6ac146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac09a970 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac0b6ef0 con 0x7fe6ac0bfc80 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b4026640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac099e90 con 0x7fe6ac0bc940 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b1dc2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6ac0b9c00 0x7fe6ac0b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b1dc2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6ac0b9c00 0x7fe6ac0b69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50282/0 (socket says 172.21.15.49:50282) 2024-03-14T22:06:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.704+0000 7fe6b1dc2640 1 -- 172.21.15.49:0/404403072 learned_addr learned my addr 172.21.15.49:0/404403072 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b25c3640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0bfc80 0x7fe6ac146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b15c1640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac1e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b1dc2640 1 -- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 msgr2=0x7fe6ac1e03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b1dc2640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac1e03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b1dc2640 1 -- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0bfc80 msgr2=0x7fe6ac146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b1dc2640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0bfc80 0x7fe6ac146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b1dc2640 1 -- 172.21.15.49:0/404403072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4009400 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b15c1640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac1e03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b1dc2640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6ac0b9c00 0x7fe6ac0b69b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400e960 tx=0x7fe6a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6a27fc640 1 -- 172.21.15.49:0/404403072 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400ca00 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c009400 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6a27fc640 1 -- 172.21.15.49:0/404403072 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe6a400cb60 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6a27fc640 1 -- 172.21.15.49:0/404403072 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4013470 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.705+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac1dfbc0 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.706+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac09a240 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.706+0000 7fe6a27fc640 1 -- 172.21.15.49:0/404403072 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe6a40135d0 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.707+0000 7fe6a27fc640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe69006a7d0 0x7fe69006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T22:06:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.707+0000 7fe6a27fc640 1 -- 172.21.15.49:0/404403072 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fe6a4089020 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.707+0000 7fe6b15c1640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe69006a7d0 0x7fe69006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T22:06:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.707+0000 7fe6b15c1640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe69006a7d0 0x7fe69006cc90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe69c036000 tx=0x7fe69c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T22:06:37.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.711+0000 7fe6a27fc640 1 -- 172.21.15.49:0/404403072 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe6a4054410 con 0x7fe6ac0b9c00 2024-03-14T22:06:37.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:37.891+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe6ac0a1d50 con 0x7fe6ac0b9c00 2024-03-14T22:06:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.543+0000 7fe6a27fc640 1 -- 172.21.15.49:0/404403072 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe6a4057a30 con 0x7fe6ac0b9c00 2024-03-14T22:06:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-03-14T22:06:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.547+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe69006a7d0 msgr2=0x7fe69006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.547+0000 7fe6b4026640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe69006a7d0 0x7fe69006cc90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe69c036000 tx=0x7fe69c008030 comp rx=0 tx=0).stop 2024-03-14T22:06:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.547+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6ac0b9c00 msgr2=0x7fe6ac0b69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T22:06:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.547+0000 7fe6b4026640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6ac0b9c00 0x7fe6ac0b69b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400e960 tx=0x7fe6a400ee20 comp rx=0 tx=0).stop 2024-03-14T22:06:38.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 shutdown_connections 2024-03-14T22:06:38.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:6832/37329,v1:172.21.15.97:6833/37329] conn(0x7fe69006a7d0 0x7fe69006cc90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:38.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6ac0bfc80 0x7fe6ac146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:38.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6ac0bc940 0x7fe6ac1e03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:38.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 --2- 172.21.15.49:0/404403072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6ac0b9c00 0x7fe6ac0b69b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T22:06:38.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 >> 172.21.15.49:0/404403072 conn(0x7fe6ac0bcda0 msgr2=0x7fe6ac0a1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T22:06:38.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 shutdown_connections 2024-03-14T22:06:38.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-14T22:06:38.549+0000 7fe6b4026640 1 -- 172.21.15.49:0/404403072 wait complete. 2024-03-14T22:06:38.568 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-14T22:06:38.568 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T22:06:38.568 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:38.580 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T22:06:38.580 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T22:06:38.580 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:38.630 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T22:06:38.630 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T22:06:38.630 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:38.644 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T22:06:38.644 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-14T22:06:38.644 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-14T22:06:38.645 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:42.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-14T22:06:44.746 INFO:tasks.ceph.osd.0:Stopped 2024-03-14T22:06:44.746 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-14T22:06:44.747 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:44.829 INFO:tasks.ceph.osd.1:Stopped 2024-03-14T22:06:44.829 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-14T22:06:44.829 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:44.903 INFO:tasks.ceph.osd.2:Stopped 2024-03-14T22:06:44.903 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-14T22:06:44.903 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:44.995 INFO:tasks.ceph.osd.3:Stopped 2024-03-14T22:06:44.995 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-14T22:06:44.995 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.080 INFO:tasks.ceph.osd.4:Stopped 2024-03-14T22:06:45.080 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-14T22:06:45.080 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.162 INFO:tasks.ceph.osd.5:Stopped 2024-03-14T22:06:45.163 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-14T22:06:45.163 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.247 INFO:tasks.ceph.osd.6:Stopped 2024-03-14T22:06:45.247 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-14T22:06:45.247 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.338 INFO:tasks.ceph.osd.7:Stopped 2024-03-14T22:06:45.338 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-14T22:06:45.339 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-14T22:06:45.339 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.365 INFO:tasks.ceph.mgr.y:Stopped 2024-03-14T22:06:45.365 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-14T22:06:45.365 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.390 INFO:tasks.ceph.mgr.x:Stopped 2024-03-14T22:06:45.390 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-14T22:06:45.390 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-14T22:06:45.390 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.418 INFO:tasks.ceph.mon.a:Stopped 2024-03-14T22:06:45.418 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-14T22:06:45.418 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.441 INFO:tasks.ceph.mon.b:Stopped 2024-03-14T22:06:45.441 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-14T22:06:45.441 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T22:06:45.492 INFO:tasks.ceph.mon.c:Stopped 2024-03-14T22:06:45.492 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-14T22:06:45.492 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-14T22:06:45.569 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T22:06:45.570 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-14T22:06:45.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T22:06:45.874 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-14T22:06:45.975 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T22:06:45.975 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-14T22:06:46.057 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T22:06:46.058 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-14T22:06:46.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T22:06:46.147 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-14T22:06:46.420 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T22:06:46.420 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-14T22:06:46.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T22:06:46.534 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-14T22:06:46.680 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T22:06:46.680 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-14T22:06:46.810 INFO:tasks.ceph:Archiving mon data... 2024-03-14T22:06:46.811 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/data/mon.a.tgz 2024-03-14T22:06:46.811 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-03-14T22:06:46.829 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.izZQPW4TIc 2024-03-14T22:06:46.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.izZQPW4TIc 2024-03-14T22:06:46.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.izZQPW4TIc 2024-03-14T22:06:47.095 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.izZQPW4TIc is 312KB 2024-03-14T22:06:47.191 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.izZQPW4TIc 2024-03-14T22:06:47.206 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/data/mon.b.tgz 2024-03-14T22:06:47.206 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-14T22:06:47.221 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.S0ejfvqPJf 2024-03-14T22:06:47.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.S0ejfvqPJf 2024-03-14T22:06:47.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.S0ejfvqPJf 2024-03-14T22:06:47.468 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.S0ejfvqPJf is 322KB 2024-03-14T22:06:47.572 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.S0ejfvqPJf 2024-03-14T22:06:47.587 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/data/mon.c.tgz 2024-03-14T22:06:47.587 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-14T22:06:47.640 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.rxg37FX5Gc 2024-03-14T22:06:47.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rxg37FX5Gc 2024-03-14T22:06:47.762 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.rxg37FX5Gc 2024-03-14T22:06:47.847 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.rxg37FX5Gc is 322KB 2024-03-14T22:06:47.942 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.rxg37FX5Gc 2024-03-14T22:06:47.956 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-14T22:06:47.956 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-14T22:06:47.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-14T22:06:47.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-14T22:06:47.998 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-14T22:06:48.097 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-14T22:06:48.098 DEBUG:teuthology.orchestra.run.smithi049:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-14T22:06:48.101 DEBUG:teuthology.orchestra.run.smithi097:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-14T22:06:48.103 DEBUG:teuthology.orchestra.run.smithi163:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-14T22:06:48.130 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-14T22:06:48.135 INFO:teuthology.orchestra.run.smithi049.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-14T22:06:48.199 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-14T22:06:48.211 INFO:tasks.ceph:Archiving crash dumps... 2024-03-14T22:06:48.213 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi049/crash 2024-03-14T22:06:48.214 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-14T22:06:48.247 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi097/crash 2024-03-14T22:06:48.248 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-14T22:06:48.275 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi163/crash 2024-03-14T22:06:48.276 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-14T22:06:48.388 INFO:tasks.ceph:Shutting down logrotate 2024-03-14T22:06:48.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:48.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:48.393 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:48.508 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:48.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:48.578 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-14T22:06:48.677 INFO:tasks.ceph:Compressing logs... 2024-03-14T22:06:48.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T22:06:48.680 DEBUG:teuthology.orchestra.run.smithi097:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T22:06:48.682 DEBUG:teuthology.orchestra.run.smithi163:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T22:09:37.812 INFO:tasks.ceph:Archiving logs... 2024-03-14T22:09:37.813 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi049/log 2024-03-14T22:09:37.814 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-14T22:11:47.686 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi097/log 2024-03-14T22:11:47.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-14T22:12:03.766 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi163/log 2024-03-14T22:12:03.767 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-14T22:12:03.868 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-14T22:12:03.878 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-14T22:12:03.878 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-14T22:12:03.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-14T22:12:03.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-14T22:12:04.042 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-14T22:12:04.051 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-14T22:12:04.051 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T22:12:04.053 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T22:12:04.056 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T22:12:04.068 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-14T22:12:04.070 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-14T22:12:04.074 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T22:12:04.075 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-14T22:12:04.075 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +3172us[+3165us] +/- 138ms 2024-03-14T22:12:04.075 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -494us[ -500us] +/- 58ms 2024-03-14T22:12:04.075 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -909us[ -916us] +/- 69ms 2024-03-14T22:12:04.075 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T22:12:04.111 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-03-14T22:12:04.114 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T22:12:04.115 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-14T22:12:04.115 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +3185us[+3185us] +/- 138ms 2024-03-14T22:12:04.115 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -486us[ -455us] +/- 58ms 2024-03-14T22:12:04.115 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -901us[ -901us] +/- 69ms 2024-03-14T22:12:04.115 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T22:12:04.121 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T22:12:04.121 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-03-14T22:12:04.121 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +3192us[+3192us] +/- 138ms 2024-03-14T22:12:04.121 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -489us[ -438us] +/- 58ms 2024-03-14T22:12:04.121 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -903us[ -853us] +/- 69ms 2024-03-14T22:12:04.121 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T22:12:04.122 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-14T22:12:04.131 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-14T22:12:04.132 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-14T22:12:04.140 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-14T22:12:04.254 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-14T22:12:04.325 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-14T22:12:04.540 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T22:12:04.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T22:12:04.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T22:12:04.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T22:12:04.614 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T22:12:04.714 INFO:teuthology.orchestra.run.smithi163.stdout:type=AVC msg=audit(1710450302.737:196): avc: denied { checkpoint_restore } for pid=1152 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-03-14T22:12:04.716 DEBUG:teuthology.task.selinux:ubuntu@smithi163.front.sepia.ceph.com has 1 denials 2024-03-14T22:12:04.716 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi163.front.sepia.ceph.com: ['type=AVC msg=audit(1710450302.737:196): avc: denied { checkpoint_restore } for pid=1152 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-14T22:12:04.717 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-14T22:12:04.727 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-14T22:12:04.743 INFO:teuthology.task.internal:Duration was 4023.032645 seconds 2024-03-14T22:12:04.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-14T22:12:04.752 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-14T22:12:04.752 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-14T22:12:04.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-14T22:12:04.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-14T22:12:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T22:12:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T22:12:04.921 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T22:12:05.259 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-14T22:12:05.259 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-03-14T22:12:05.260 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-14T22:12:05.283 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-03-14T22:12:05.283 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-14T22:12:05.315 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-03-14T22:12:05.315 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-14T22:12:05.394 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-14T22:12:05.395 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T22:12:05.400 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T22:12:05.405 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T22:12:05.495 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-14T22:12:05.496 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-14T22:12:05.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-14T22:12:05.504 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-14T22:12:06.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-14T22:12:06.393 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-14T22:12:06.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-14T22:12:06.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-14T22:12:06.473 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-14T22:12:06.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-14T22:12:06.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-14T22:12:06.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-14T22:12:06.582 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-14T22:12:06.609 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-03-14T22:12:06.630 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-03-14T22:12:06.688 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-03-14T22:12:06.742 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-14T22:12:06.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T22:12:06.759 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-14T22:12:06.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T22:12:06.776 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-14T22:12:06.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T22:12:06.830 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-14T22:12:06.840 INFO:teuthology.task.internal:Transferring archived files... 2024-03-14T22:12:06.841 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi049 2024-03-14T22:12:06.841 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-14T22:12:06.885 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi097 2024-03-14T22:12:06.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-14T22:12:06.928 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405/remote/smithi163 2024-03-14T22:12:06.929 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-14T22:12:07.040 INFO:teuthology.task.internal:Removing archive directory... 2024-03-14T22:12:07.041 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-14T22:12:07.043 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-14T22:12:07.046 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-14T22:12:07.103 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-14T22:12:07.114 INFO:teuthology.task.internal:Not uploading archives. 2024-03-14T22:12:07.114 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-14T22:12:07.123 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-14T22:12:07.123 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-14T22:12:07.126 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-14T22:12:07.128 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-14T22:12:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 14 22:12 /home/ubuntu/cephtest 2024-03-14T22:12:07.143 INFO:teuthology.orchestra.run.smithi097.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 14 22:12 /home/ubuntu/cephtest 2024-03-14T22:12:07.184 INFO:teuthology.orchestra.run.smithi163.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 14 22:12 /home/ubuntu/cephtest 2024-03-14T22:12:07.187 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-14T22:12:07.198 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-14T22:12:07.208 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi163.front.sepia.ceph.com: ['type=AVC msg=audit(1710450302.737:196): avc: denied { checkpoint_restore } for pid=1152 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-14T22:12:07.209 INFO:teuthology.nuke:Checking targets against current locks 2024-03-14T22:12:07.232 DEBUG:teuthology.nuke:shortname: smithi049 2024-03-14T22:12:07.233 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T22:12:07.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 20:54:34.964322', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIsFzqFfZuzGWdRz7NTpDc8OGmCraRPPVX0whB8j8nGlLW2CZFDOrfhSltO7o0I8PMZJ5UojN0FCwrTlpMIbjJA='} 2024-03-14T22:12:07.258 DEBUG:teuthology.nuke:shortname: smithi097 2024-03-14T22:12:07.259 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T22:12:07.280 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 20:54:34.962660', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC8FSt3cq8kt44xgUYd26EGx1zagHHuP61WQJ0TefnB9toTrZAsEFMYq1EvrHvwIBAw+dx4UMwj6Tg5nT+YNDVY='} 2024-03-14T22:12:07.284 DEBUG:teuthology.nuke:shortname: smithi163 2024-03-14T22:12:07.285 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T22:12:07.305 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 20:54:34.960779', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvawradYzO1Z9o2QO+uZ+V6JJs7yGcUFYNr2bd+3nXey6xgvubX6cevWGSStxIMBct5lZ/ACj4rJ4eA1VvS/8Y='} 2024-03-14T22:12:07.314 INFO:teuthology.orchestra.console.smithi049:Power off 2024-03-14T22:12:07.314 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T22:12:07.338 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2024-03-14T22:12:07.338 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:07.355 INFO:teuthology.orchestra.console.smithi097:Power off 2024-03-14T22:12:07.355 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T22:12:07.366 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-14T22:12:07.377 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-03-14T22:12:07.377 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:07.391 INFO:teuthology.orchestra.console.smithi163:Power off 2024-03-14T22:12:07.391 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T22:12:07.400 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-14T22:12:07.412 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-03-14T22:12:07.412 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:07.434 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-14T22:12:11.367 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:11.401 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:11.435 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:11.478 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-14T22:12:11.510 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-14T22:12:11.545 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-14T22:12:15.481 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:15.511 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:15.546 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:15.591 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-14T22:12:15.621 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-03-14T22:12:15.655 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-03-14T22:12:15.722 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-03-14T22:12:15.756 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-03-14T22:12:19.593 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T22:12:19.702 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2024-03-14T22:12:19.803 INFO:teuthology.orchestra.console.smithi049:Power off completed 2024-03-14T22:12:19.908 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3845b7ac715a3ff198582638d199a90e30e254da description: fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-dbench} duration: 4023.032645225525 failure_reason: 'SELinux denials found on ubuntu@smithi163.front.sepia.ceph.com: [''type=AVC msg=audit(1710450302.737:196): avc: denied { checkpoint_restore } for pid=1152 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'']' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-03-14T22:12:19.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T22:12:19.986 INFO:teuthology.run:FAIL